2024-09-11T07:56:12.678 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T07:56:12.693 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T07:56:12.779 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901259 branch: wip-guits-quincy-2024-09-10-2008-quincy description: orch:cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} email: gabrioux@ibm.com first_in_suite: false job_id: '7901259' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-10-2008-quincy ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: false mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - MON_DOWN - mons down - mon down - out of quorum sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 kernel: hwe: true selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-quincy-2024-09-10-2008-quincy sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 2282 sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-quincy-2024-09-10-2008-quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 targets: smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOSZqKEIb2jRAhkRWHTkyDumCsZbEbbJhYa5h7nXqEgb3QRp4AqfE68ojfTixPoEOjD3QXN0vtAwJNRCJyxFvRY= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBRhrPqAcR/oA0a7E7OKRdwZcUiHIIhPqhIVywXymbUoO39M3o9+MFqe6dPvi1R4ZyyYJu5dJfHunXN2WH1eZw8= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_06:53:43 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T07:56:12.780 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa; will attempt to use it 2024-09-11T07:56:12.780 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa/tasks 2024-09-11T07:56:12.781 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T07:56:12.782 INFO:teuthology.task.internal:Checking packages... 2024-09-11T07:56:12.806 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd0e6828a2016d48cf25ad84064e50742bb1c39b9' 2024-09-11T07:56:12.806 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T07:56:12.806 INFO:teuthology.packaging:ref: None 2024-09-11T07:56:12.806 INFO:teuthology.packaging:tag: None 2024-09-11T07:56:12.806 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T07:56:12.806 INFO:teuthology.packaging:sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:56:12.807 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T07:56:12.956 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1623.gd0e6828a 2024-09-11T07:56:12.958 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T07:56:12.965 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T07:56:12.965 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T07:56:13.025 INFO:teuthology.task.internal:Saving configuration 2024-09-11T07:56:13.036 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T07:56:13.042 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T07:56:13.070 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901259', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 07:51:17.905093', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOSZqKEIb2jRAhkRWHTkyDumCsZbEbbJhYa5h7nXqEgb3QRp4AqfE68ojfTixPoEOjD3QXN0vtAwJNRCJyxFvRY='} 2024-09-11T07:56:13.090 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901259', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 07:51:17.903753', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBRhrPqAcR/oA0a7E7OKRdwZcUiHIIhPqhIVywXymbUoO39M3o9+MFqe6dPvi1R4ZyyYJu5dJfHunXN2WH1eZw8='} 2024-09-11T07:56:13.090 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T07:56:13.096 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-11T07:56:13.096 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-11T07:56:13.096 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T07:56:13.261 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f83b49e7ac0>, signals=[15]) 2024-09-11T07:56:13.261 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T07:56:13.267 INFO:teuthology.task.internal:Opening connections... 2024-09-11T07:56:13.267 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-09-11T07:56:13.269 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:56:13.345 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-09-11T07:56:13.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:56:13.419 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T07:56:13.425 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-09-11T07:56:13.444 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-09-11T07:56:13.445 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-09-11T07:56:13.502 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2024-09-11T07:56:13.503 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="9" 2024-09-11T07:56:13.503 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2024-09-11T07:56:13.503 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2024-09-11T07:56:13.503 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="9" 2024-09-11T07:56:13.503 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el9" 2024-09-11T07:56:13.503 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T07:56:13.503 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2024-09-11T07:56:13.503 INFO:teuthology.orchestra.run.smithi071.stdout:LOGO="fedora-logo-icon" 2024-09-11T07:56:13.503 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T07:56:13.503 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2024-09-11T07:56:13.503 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T07:56:13.504 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T07:56:13.504 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T07:56:13.504 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-09-11T07:56:13.529 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-09-11T07:56:13.547 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-09-11T07:56:13.547 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-09-11T07:56:13.605 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-09-11T07:56:13.606 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-09-11T07:56:13.606 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-09-11T07:56:13.606 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-09-11T07:56:13.606 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-09-11T07:56:13.606 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-09-11T07:56:13.606 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T07:56:13.606 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-09-11T07:56:13.606 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-09-11T07:56:13.606 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T07:56:13.606 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-09-11T07:56:13.606 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T07:56:13.606 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T07:56:13.607 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T07:56:13.607 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-09-11T07:56:13.628 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T07:56:13.638 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T07:56:13.662 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T07:56:13.662 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-09-11T07:56:13.664 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-09-11T07:56:13.682 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T07:56:13.688 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T07:56:13.688 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-09-11T07:56:13.721 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-09-11T07:56:13.792 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T07:56:13.845 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T07:56:13.860 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T07:56:13.860 INFO:teuthology.task.kernel:normalize config orig: {'hwe': True} 2024-09-11T07:56:13.860 DEBUG:teuthology.task.kernel:normalized overrides {'mon.a': {'hwe': True}, 'mon.c': {'hwe': True}, 'mgr.y': {'hwe': True}, 'osd.0': {'hwe': True}, 'osd.1': {'hwe': True}, 'osd.2': {'hwe': True}, 'osd.3': {'hwe': True}, 'client.0': {'hwe': True}, 'ceph.rgw.foo.a': {'hwe': True}, 'node-exporter.a': {'hwe': True}, 'alertmanager.a': {'hwe': True}, 'mon.b': {'hwe': True}, 'mgr.x': {'hwe': True}, 'osd.4': {'hwe': True}, 'osd.5': {'hwe': True}, 'osd.6': {'hwe': True}, 'osd.7': {'hwe': True}, 'client.1': {'hwe': True}, 'prometheus.a': {'hwe': True}, 'grafana.a': {'hwe': True}, 'node-exporter.b': {'hwe': True}, 'ceph.iscsi.iscsi.a': {'hwe': True}} 2024-09-11T07:56:13.861 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro', 'hwe': True}, 'mon.b': {'kdb': 1, 'sha1': 'distro', 'hwe': True}}, timeout 300 2024-09-11T07:56:13.861 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T07:56:13.861 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T07:56:13.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:56:13.879 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-09-11T07:56:13.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:56:13.880 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-09-11T07:56:13.936 INFO:teuthology.orchestra.run.smithi071.stdout:5.14.0-503.el9.x86_64 2024-09-11T07:56:13.937 INFO:teuthology.task.kernel:Running kernel on smithi071: 5.14.0-503.el9.x86_64 2024-09-11T07:56:13.937 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum install -y kernel 2024-09-11T07:56:13.938 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-503.el9.x86_64 2024-09-11T07:56:13.938 INFO:teuthology.task.kernel:Running kernel on smithi196: 5.14.0-503.el9.x86_64 2024-09-11T07:56:13.938 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum install -y kernel 2024-09-11T07:56:16.358 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-11T07:56:17.490 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-09-11T07:56:20.908 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 6.0 MB/s | 20 MB 00:03 2024-09-11T07:56:22.799 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 4.8 MB/s | 20 MB 00:04 2024-09-11T07:56:26.284 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.5 MB 00:01 2024-09-11T07:56:28.024 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.5 MB 00:01 2024-09-11T07:56:28.667 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-11T07:56:29.463 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-11T07:56:29.541 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 19 kB 00:00 2024-09-11T07:56:30.313 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-11T07:56:34.927 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-11T07:56:35.759 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-11T07:56:36.891 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T07:56:36.892 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T07:56:36.973 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-11T07:56:36.980 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T07:56:36.980 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-09-11T07:56:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T07:56:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-09-11T07:56:36.981 INFO:teuthology.orchestra.run.smithi196.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T07:56:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-09-11T07:56:36.981 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T07:56:36.981 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T07:56:36.982 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T07:56:36.982 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:56:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-11T07:56:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T07:56:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:Install 4 Packages 2024-09-11T07:56:36.982 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:56:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 79 M 2024-09-11T07:56:36.983 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 126 M 2024-09-11T07:56:36.983 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-09-11T07:56:37.228 INFO:teuthology.orchestra.run.smithi196.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 178 kB/s | 23 kB 00:00 2024-09-11T07:56:37.774 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T07:56:37.774 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T07:56:37.871 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-11T07:56:37.875 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T07:56:37.875 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-09-11T07:56:37.876 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T07:56:37.876 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-11T07:56:37.876 INFO:teuthology.orchestra.run.smithi071.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T07:56:37.876 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-09-11T07:56:37.876 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T07:56:37.876 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T07:56:37.876 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T07:56:37.876 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:56:37.876 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-11T07:56:37.876 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T07:56:37.876 INFO:teuthology.orchestra.run.smithi071.stdout:Install 4 Packages 2024-09-11T07:56:37.876 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:56:37.877 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 79 M 2024-09-11T07:56:37.877 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 126 M 2024-09-11T07:56:37.877 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-11T07:56:38.145 INFO:teuthology.orchestra.run.smithi196.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 31 MB/s | 29 MB 00:00 2024-09-11T07:56:38.221 INFO:teuthology.orchestra.run.smithi071.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 77 kB/s | 23 kB 00:00 2024-09-11T07:56:39.003 INFO:teuthology.orchestra.run.smithi196.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 8.2 MB/s | 16 MB 00:01 2024-09-11T07:56:39.596 INFO:teuthology.orchestra.run.smithi196.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 14 MB/s | 35 MB 00:02 2024-09-11T07:56:39.596 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:56:39.596 INFO:teuthology.orchestra.run.smithi196.stdout:Total 30 MB/s | 79 MB 00:02 2024-09-11T07:56:40.275 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-11T07:56:40.634 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-11T07:56:40.634 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-11T07:56:41.607 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-11T07:56:41.607 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-11T07:56:41.727 INFO:teuthology.orchestra.run.smithi071.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 8.1 MB/s | 29 MB 00:03 2024-09-11T07:56:44.056 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-11T07:56:44.796 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:56:44.880 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:56:46.835 INFO:teuthology.orchestra.run.smithi071.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 1.8 MB/s | 16 MB 00:08 2024-09-11T07:56:47.663 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:56:47.790 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:56:49.368 INFO:teuthology.orchestra.run.smithi071.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 3.0 MB/s | 35 MB 00:11 2024-09-11T07:56:49.368 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:56:49.368 INFO:teuthology.orchestra.run.smithi071.stdout:Total 6.9 MB/s | 79 MB 00:11 2024-09-11T07:56:50.052 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-11T07:56:50.399 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-11T07:56:50.399 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-11T07:56:51.361 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-11T07:56:51.361 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-11T07:56:52.629 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:56:52.741 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:56:53.777 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-11T07:56:54.446 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:56:54.528 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:56:57.318 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:56:57.458 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:56:57.613 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:57:02.286 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:57:02.400 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:57:07.259 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:57:35.858 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:57:35.898 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:57:36.257 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:57:36.258 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:57:36.258 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:57:36.258 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:57:37.108 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:57:37.109 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:57:37.109 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-09-11T07:57:37.109 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T07:57:37.109 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T07:57:37.109 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T07:57:37.109 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T07:57:37.109 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:57:37.109 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-11T07:57:37.467 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:57:37.561 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:57:37.561 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:57:37.561 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T07:57:37.561 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T07:57:37.561 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T07:57:37.561 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-11T07:57:37.562 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi196.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T07:57:37.562 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:57:37.620 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:57:37.620 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:57:37.620 DEBUG:teuthology.orchestra.run.smithi196:> sudo rpm -qi grub2-tools 2024-09-11T07:57:37.652 INFO:teuthology.orchestra.run.smithi196.stdout:Name : grub2-tools 2024-09-11T07:57:37.653 INFO:teuthology.orchestra.run.smithi196.stdout:Epoch : 1 2024-09-11T07:57:37.653 INFO:teuthology.orchestra.run.smithi196.stdout:Version : 2.06 2024-09-11T07:57:37.653 INFO:teuthology.orchestra.run.smithi196.stdout:Release : 82.el9 2024-09-11T07:57:37.653 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: x86_64 2024-09-11T07:57:37.653 INFO:teuthology.orchestra.run.smithi196.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T07:57:37.653 INFO:teuthology.orchestra.run.smithi196.stdout:Group : Unspecified 2024-09-11T07:57:37.653 INFO:teuthology.orchestra.run.smithi196.stdout:Size : 8274670 2024-09-11T07:57:37.653 INFO:teuthology.orchestra.run.smithi196.stdout:License : GPLv3+ 2024-09-11T07:57:37.653 INFO:teuthology.orchestra.run.smithi196.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T07:57:37.653 INFO:teuthology.orchestra.run.smithi196.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T07:57:37.653 INFO:teuthology.orchestra.run.smithi196.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T07:57:37.653 INFO:teuthology.orchestra.run.smithi196.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T07:57:37.654 INFO:teuthology.orchestra.run.smithi196.stdout:Packager : builder@centos.org 2024-09-11T07:57:37.654 INFO:teuthology.orchestra.run.smithi196.stdout:Vendor : CentOS 2024-09-11T07:57:37.654 INFO:teuthology.orchestra.run.smithi196.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T07:57:37.654 INFO:teuthology.orchestra.run.smithi196.stdout:Summary : Support tools for GRUB. 2024-09-11T07:57:37.654 INFO:teuthology.orchestra.run.smithi196.stdout:Description : 2024-09-11T07:57:37.654 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:57:37.654 INFO:teuthology.orchestra.run.smithi196.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T07:57:37.654 INFO:teuthology.orchestra.run.smithi196.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T07:57:37.654 INFO:teuthology.orchestra.run.smithi196.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T07:57:37.654 INFO:teuthology.orchestra.run.smithi196.stdout:hardware devices. 2024-09-11T07:57:37.654 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:57:37.654 INFO:teuthology.orchestra.run.smithi196.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T07:57:37.655 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T07:57:37.655 INFO:teuthology.task.kernel:Updating grub on smithi196 to boot 5.14.0-505.el9.x86_64 2024-09-11T07:57:37.655 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T07:57:38.326 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2024-09-11T07:57:39.638 INFO:teuthology.orchestra.run.smithi196.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T07:57:39.670 INFO:teuthology.orchestra.run.smithi196.stderr:done 2024-09-11T07:57:39.673 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-09-11T07:57:39.689 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.iLiv1HN7Ux 2024-09-11T07:57:39.689 DEBUG:teuthology.orchestra.run.smithi196:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.iLiv1HN7Ux 2024-09-11T07:57:39.757 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.iLiv1HN7Ux 2024-09-11T07:57:39.898 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.iLiv1HN7Ux is 6KB 2024-09-11T07:57:39.945 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.iLiv1HN7Ux 2024-09-11T07:57:39.960 DEBUG:teuthology.orchestra.run.smithi196:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T07:57:40.023 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T07:57:40.023 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T07:57:40.023 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T07:57:40.024 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T07:57:40.025 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T07:57:40.172 DEBUG:teuthology.orchestra.run.smithi196:> sudo shutdown -r now 2024-09-11T07:57:45.322 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:57:45.363 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:57:45.712 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:57:45.713 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:57:45.713 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:57:45.713 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:57:46.541 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:57:46.541 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:57:46.541 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-11T07:57:46.541 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T07:57:46.541 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T07:57:46.541 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T07:57:46.542 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T07:57:46.542 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:57:46.542 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-11T07:57:46.909 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:57:47.043 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:57:47.044 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:57:47.044 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T07:57:47.044 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T07:57:47.044 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T07:57:47.044 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T07:57:47.044 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi071.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T07:57:47.044 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:57:47.104 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:57:47.104 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:57:47.104 DEBUG:teuthology.orchestra.run.smithi071:> sudo rpm -qi grub2-tools 2024-09-11T07:57:47.136 INFO:teuthology.orchestra.run.smithi071.stdout:Name : grub2-tools 2024-09-11T07:57:47.136 INFO:teuthology.orchestra.run.smithi071.stdout:Epoch : 1 2024-09-11T07:57:47.136 INFO:teuthology.orchestra.run.smithi071.stdout:Version : 2.06 2024-09-11T07:57:47.136 INFO:teuthology.orchestra.run.smithi071.stdout:Release : 82.el9 2024-09-11T07:57:47.136 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: x86_64 2024-09-11T07:57:47.136 INFO:teuthology.orchestra.run.smithi071.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T07:57:47.136 INFO:teuthology.orchestra.run.smithi071.stdout:Group : Unspecified 2024-09-11T07:57:47.136 INFO:teuthology.orchestra.run.smithi071.stdout:Size : 8274670 2024-09-11T07:57:47.137 INFO:teuthology.orchestra.run.smithi071.stdout:License : GPLv3+ 2024-09-11T07:57:47.137 INFO:teuthology.orchestra.run.smithi071.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T07:57:47.137 INFO:teuthology.orchestra.run.smithi071.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T07:57:47.137 INFO:teuthology.orchestra.run.smithi071.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T07:57:47.137 INFO:teuthology.orchestra.run.smithi071.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T07:57:47.137 INFO:teuthology.orchestra.run.smithi071.stdout:Packager : builder@centos.org 2024-09-11T07:57:47.137 INFO:teuthology.orchestra.run.smithi071.stdout:Vendor : CentOS 2024-09-11T07:57:47.137 INFO:teuthology.orchestra.run.smithi071.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T07:57:47.137 INFO:teuthology.orchestra.run.smithi071.stdout:Summary : Support tools for GRUB. 2024-09-11T07:57:47.137 INFO:teuthology.orchestra.run.smithi071.stdout:Description : 2024-09-11T07:57:47.137 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:57:47.137 INFO:teuthology.orchestra.run.smithi071.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T07:57:47.137 INFO:teuthology.orchestra.run.smithi071.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T07:57:47.137 INFO:teuthology.orchestra.run.smithi071.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T07:57:47.138 INFO:teuthology.orchestra.run.smithi071.stdout:hardware devices. 2024-09-11T07:57:47.138 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:57:47.138 INFO:teuthology.orchestra.run.smithi071.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T07:57:47.138 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T07:57:47.139 INFO:teuthology.task.kernel:Updating grub on smithi071 to boot 5.14.0-505.el9.x86_64 2024-09-11T07:57:47.139 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T07:57:47.860 INFO:teuthology.orchestra.run.smithi071.stderr:Generating grub configuration file ... 2024-09-11T07:57:49.178 INFO:teuthology.orchestra.run.smithi071.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T07:57:49.224 INFO:teuthology.orchestra.run.smithi071.stderr:done 2024-09-11T07:57:49.226 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-09-11T07:57:49.241 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.oSRq0Ye1rV 2024-09-11T07:57:49.241 DEBUG:teuthology.orchestra.run.smithi071:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.oSRq0Ye1rV 2024-09-11T07:57:49.313 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.oSRq0Ye1rV 2024-09-11T07:57:49.450 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.oSRq0Ye1rV is 6KB 2024-09-11T07:57:49.499 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.oSRq0Ye1rV 2024-09-11T07:57:49.514 DEBUG:teuthology.orchestra.run.smithi071:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T07:57:49.576 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T07:57:49.577 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T07:57:49.577 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T07:57:49.577 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T07:57:49.578 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T07:57:49.730 DEBUG:teuthology.orchestra.run.smithi071:> sudo shutdown -r now 2024-09-11T07:58:10.194 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-11T07:58:10.194 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-11T07:58:10.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:58:19.741 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T07:58:19.741 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T07:58:19.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:58:54.269 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-09-11T07:59:03.277 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T07:59:03.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:59:10.204 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T07:59:19.213 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-11T07:59:19.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:59:19.498 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-09-11T07:59:20.198 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-11T07:59:20.198 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-11T07:59:20.199 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-09-11T07:59:20.214 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-505.el9.x86_64 2024-09-11T07:59:20.214 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T07:59:20.214 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T07:59:20.214 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T07:59:21.215 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-11T07:59:21.215 DEBUG:teuthology.orchestra.run.smithi196:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T07:59:21.299 INFO:teuthology.orchestra.run.smithi196.stdout:ttyS1 2024-09-11T07:59:21.328 DEBUG:teuthology.parallel:result is None 2024-09-11T07:59:21.661 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-09-11T07:59:33.662 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T07:59:33.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:59:34.080 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-09-11T07:59:34.421 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T07:59:34.422 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-11T07:59:34.422 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-09-11T07:59:34.436 INFO:teuthology.orchestra.run.smithi071.stdout:5.14.0-505.el9.x86_64 2024-09-11T07:59:34.436 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T07:59:34.437 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T07:59:34.437 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T07:59:35.438 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T07:59:35.438 DEBUG:teuthology.orchestra.run.smithi071:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T07:59:35.532 INFO:teuthology.orchestra.run.smithi071.stdout:ttyS1 2024-09-11T07:59:35.555 DEBUG:teuthology.parallel:result is None 2024-09-11T07:59:35.555 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T07:59:35.564 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T07:59:35.564 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T07:59:35.566 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T07:59:35.587 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T07:59:35.593 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T07:59:35.600 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T07:59:35.601 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T07:59:35.621 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T07:59:35.663 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T07:59:35.670 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T07:59:35.670 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T07:59:35.686 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T07:59:35.709 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:59:35.718 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:59:35.727 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:59:35.738 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:59:35.740 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T07:59:35.747 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T07:59:35.747 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T07:59:35.762 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T07:59:35.808 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T07:59:35.831 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T07:59:35.831 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T07:59:35.833 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T07:59:35.864 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:59:35.929 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:59:36.011 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:59:36.051 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:59:36.092 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:59:36.092 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T07:59:36.156 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:59:36.202 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:59:36.248 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:59:36.332 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:59:36.371 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T07:59:36.371 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T07:59:36.437 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-09-11T07:59:36.446 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-09-11T07:59:36.506 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:59:36.531 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:59:36.968 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T07:59:36.976 INFO:teuthology.task.internal:Starting timer... 2024-09-11T07:59:36.976 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T07:59:37.010 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T07:59:37.019 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-11T07:59:37.019 DEBUG:teuthology.orchestra.run.smithi071:> sudo service auditd rotate 2024-09-11T07:59:37.144 INFO:teuthology.orchestra.run.smithi071.stdout:Rotating logs: 2024-09-11T07:59:37.148 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-09-11T07:59:37.198 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-09-11T07:59:37.200 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T07:59:37.200 DEBUG:teuthology.orchestra.run.smithi071:> /usr/sbin/getenforce 2024-09-11T07:59:37.266 INFO:teuthology.orchestra.run.smithi071.stdout:Permissive 2024-09-11T07:59:37.267 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-09-11T07:59:37.291 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-09-11T07:59:37.291 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi071.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-09-11T07:59:37.292 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:59:37.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:59:37.381 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:59:37.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:59:37.408 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T07:59:37.408 DEBUG:teuthology.orchestra.run.smithi071:> sudo /usr/sbin/setenforce permissive 2024-09-11T07:59:37.512 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-09-11T07:59:37.549 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T07:59:37.558 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-11T07:59:37.562 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-11T07:59:37.674 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T07:59:37.685 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T07:59:37.686 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi071.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T08:03:20.850 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi071.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-09-11T08:03:20.852 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T08:03:20.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:03:20.929 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-09-11T08:03:21.007 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T08:03:21.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-11T08:03:21.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:03:21.080 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-09-11T08:03:21.156 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-11T08:03:21.156 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T08:03:21.169 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T08:03:21.169 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T08:03:21.170 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:03:21.172 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T08:03:21.172 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:03:21.199 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T08:03:21.215 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T08:03:21.227 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T08:03:21.239 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: ntpd: command not found 2024-09-11T08:03:21.241 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T08:03:21.251 INFO:teuthology.orchestra.run.smithi071.stdout:506 Cannot talk to daemon 2024-09-11T08:03:21.265 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T08:03:21.266 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-09-11T08:03:21.277 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-09-11T08:03:21.279 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T08:03:21.292 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T08:03:21.306 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T08:03:21.337 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:03:21.341 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:03:21.341 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-09-11T08:03:21.341 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:03:21.341 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:03:21.342 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:03:21.342 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:03:21.361 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:03:21.364 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:03:21.365 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-09-11T08:03:21.365 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:03:21.365 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:03:21.365 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:03:21.365 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:03:21.366 INFO:teuthology.run_tasks:Running task pexec... 2024-09-11T08:03:21.375 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-11T08:03:21.375 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi071.front.sepia.ceph.com 2024-09-11T08:03:21.375 DEBUG:teuthology.orchestra.run.smithi071:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T08:03:21.376 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi196.front.sepia.ceph.com 2024-09-11T08:03:21.376 DEBUG:teuthology.orchestra.run.smithi196:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T08:03:21.730 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-11T08:03:21.738 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T08:03:21.738 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-09-11T08:03:21.738 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T08:03:21.738 INFO:teuthology.orchestra.run.smithi071.stdout:Removing: 2024-09-11T08:03:21.738 INFO:teuthology.orchestra.run.smithi071.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T08:03:21.738 INFO:teuthology.orchestra.run.smithi071.stdout:Removing unused dependencies: 2024-09-11T08:03:21.738 INFO:teuthology.orchestra.run.smithi071.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T08:03:21.738 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:03:21.738 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-11T08:03:21.738 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T08:03:21.739 INFO:teuthology.orchestra.run.smithi071.stdout:Remove 2 Packages 2024-09-11T08:03:21.739 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:03:21.739 INFO:teuthology.orchestra.run.smithi071.stdout:Freed space: 5.5 M 2024-09-11T08:03:21.739 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-11T08:03:21.742 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-11T08:03:21.743 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-11T08:03:21.755 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-11T08:03:21.769 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T08:03:21.769 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-09-11T08:03:21.769 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T08:03:21.769 INFO:teuthology.orchestra.run.smithi196.stdout:Removing: 2024-09-11T08:03:21.770 INFO:teuthology.orchestra.run.smithi196.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T08:03:21.770 INFO:teuthology.orchestra.run.smithi196.stdout:Removing unused dependencies: 2024-09-11T08:03:21.770 INFO:teuthology.orchestra.run.smithi196.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T08:03:21.770 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T08:03:21.770 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-11T08:03:21.770 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T08:03:21.770 INFO:teuthology.orchestra.run.smithi196.stdout:Remove 2 Packages 2024-09-11T08:03:21.770 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T08:03:21.770 INFO:teuthology.orchestra.run.smithi196.stdout:Freed space: 5.5 M 2024-09-11T08:03:21.770 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-11T08:03:21.774 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-11T08:03:21.774 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-11T08:03:21.811 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-11T08:03:21.812 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-11T08:03:21.826 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-11T08:03:21.827 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-11T08:03:21.953 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-11T08:03:21.969 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-11T08:03:22.033 INFO:teuthology.orchestra.run.smithi071.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T08:03:22.041 INFO:teuthology.orchestra.run.smithi196.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T08:03:22.097 INFO:teuthology.orchestra.run.smithi071.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T08:03:22.102 INFO:teuthology.orchestra.run.smithi196.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T08:03:22.452 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T08:03:22.452 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T08:03:22.593 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T08:03:22.594 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T08:03:22.690 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T08:03:22.690 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:03:22.690 INFO:teuthology.orchestra.run.smithi071.stdout:Removed: 2024-09-11T08:03:22.691 INFO:teuthology.orchestra.run.smithi071.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T08:03:22.691 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:03:22.691 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-11T08:03:22.870 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T08:03:22.870 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T08:03:22.871 INFO:teuthology.orchestra.run.smithi196.stdout:Removed: 2024-09-11T08:03:22.871 INFO:teuthology.orchestra.run.smithi196.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T08:03:22.871 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T08:03:22.871 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-11T08:03:23.547 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:51 ago on Wed 11 Sep 2024 08:02:32 AM UTC. 2024-09-11T08:03:23.678 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:00:51 ago on Wed 11 Sep 2024 08:02:32 AM UTC. 2024-09-11T08:03:23.685 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-11T08:03:23.686 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T08:03:23.687 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-09-11T08:03:23.687 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T08:03:23.687 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-11T08:03:23.687 INFO:teuthology.orchestra.run.smithi071.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T08:03:23.687 INFO:teuthology.orchestra.run.smithi071.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T08:03:23.687 INFO:teuthology.orchestra.run.smithi071.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-11T08:03:23.687 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-09-11T08:03:23.687 INFO:teuthology.orchestra.run.smithi071.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T08:03:23.687 INFO:teuthology.orchestra.run.smithi071.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T08:03:23.687 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T08:03:23.687 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T08:03:23.687 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:03:23.687 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-11T08:03:23.687 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T08:03:23.688 INFO:teuthology.orchestra.run.smithi071.stdout:Install 7 Packages 2024-09-11T08:03:23.688 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:03:23.711 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 5.2 M 2024-09-11T08:03:23.711 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 19 M 2024-09-11T08:03:23.711 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-11T08:03:23.815 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-11T08:03:23.816 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T08:03:23.816 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-09-11T08:03:23.816 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T08:03:23.816 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-09-11T08:03:23.816 INFO:teuthology.orchestra.run.smithi196.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T08:03:23.816 INFO:teuthology.orchestra.run.smithi196.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T08:03:23.816 INFO:teuthology.orchestra.run.smithi196.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-11T08:03:23.817 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-09-11T08:03:23.817 INFO:teuthology.orchestra.run.smithi196.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T08:03:23.817 INFO:teuthology.orchestra.run.smithi196.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T08:03:23.817 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T08:03:23.817 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T08:03:23.817 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T08:03:23.817 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-11T08:03:23.817 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T08:03:23.817 INFO:teuthology.orchestra.run.smithi196.stdout:Install 7 Packages 2024-09-11T08:03:23.817 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T08:03:23.829 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 5.2 M 2024-09-11T08:03:23.830 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 19 M 2024-09-11T08:03:23.830 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-09-11T08:03:24.126 INFO:teuthology.orchestra.run.smithi071.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 428 kB/s | 44 kB 00:00 2024-09-11T08:03:24.193 INFO:teuthology.orchestra.run.smithi071.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 620 kB/s | 106 kB 00:00 2024-09-11T08:03:24.219 INFO:teuthology.orchestra.run.smithi071.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 779 kB/s | 72 kB 00:00 2024-09-11T08:03:24.244 INFO:teuthology.orchestra.run.smithi071.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-11T08:03:24.249 INFO:teuthology.orchestra.run.smithi196.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 476 kB/s | 44 kB 00:00 2024-09-11T08:03:24.283 INFO:teuthology.orchestra.run.smithi196.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 835 kB/s | 106 kB 00:00 2024-09-11T08:03:24.316 INFO:teuthology.orchestra.run.smithi196.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 1.0 MB/s | 72 kB 00:00 2024-09-11T08:03:24.350 INFO:teuthology.orchestra.run.smithi196.stdout:(4/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 4.6 MB/s | 906 kB 00:00 2024-09-11T08:03:24.369 INFO:teuthology.orchestra.run.smithi071.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.6 MB/s | 906 kB 00:00 2024-09-11T08:03:24.375 INFO:teuthology.orchestra.run.smithi196.stdout:(5/7): python3-kmod-0.9-32.el9.x86_64.rpm 915 kB/s | 84 kB 00:00 2024-09-11T08:03:24.433 INFO:teuthology.orchestra.run.smithi196.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 7.0 MB/s | 837 kB 00:00 2024-09-11T08:03:24.436 INFO:teuthology.orchestra.run.smithi071.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 3.8 MB/s | 837 kB 00:00 2024-09-11T08:03:24.561 INFO:teuthology.orchestra.run.smithi071.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 10 MB/s | 3.2 MB 00:00 2024-09-11T08:03:24.562 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:03:24.562 INFO:teuthology.orchestra.run.smithi071.stdout:Total 6.1 MB/s | 5.2 MB 00:00 2024-09-11T08:03:24.671 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-11T08:03:24.686 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-11T08:03:24.686 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-11T08:03:25.009 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-11T08:03:25.010 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-11T08:03:25.317 INFO:teuthology.orchestra.run.smithi196.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 3.3 MB/s | 3.2 MB 00:00 2024-09-11T08:03:25.318 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:03:25.318 INFO:teuthology.orchestra.run.smithi196.stdout:Total 3.5 MB/s | 5.2 MB 00:01 2024-09-11T08:03:25.435 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-11T08:03:25.450 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-11T08:03:25.450 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-11T08:03:25.496 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-11T08:03:25.588 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-11T08:03:25.634 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-11T08:03:25.671 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-11T08:03:25.758 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-11T08:03:25.759 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-11T08:03:25.879 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-11T08:03:25.927 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T08:03:26.225 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-11T08:03:26.303 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-11T08:03:26.338 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-11T08:03:26.366 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-11T08:03:26.398 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T08:03:26.429 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T08:03:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T08:03:26.575 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-11T08:03:26.606 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T08:03:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:03:27.042 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T08:03:27.067 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T08:03:27.188 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T08:03:27.253 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:03:27.629 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:03:27.629 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-11T08:03:27.629 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-11T08:03:27.629 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-11T08:03:27.629 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-11T08:03:27.629 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-11T08:03:27.629 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-11T08:03:28.023 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:03:28.023 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:03:28.023 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-11T08:03:28.023 INFO:teuthology.orchestra.run.smithi071.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T08:03:28.023 INFO:teuthology.orchestra.run.smithi071.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T08:03:28.023 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T08:03:28.023 INFO:teuthology.orchestra.run.smithi071.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-11T08:03:28.023 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:03:28.023 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-11T08:03:28.215 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:03:28.215 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-11T08:03:28.216 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-11T08:03:28.216 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-11T08:03:28.216 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-11T08:03:28.216 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-11T08:03:28.216 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-11T08:03:28.410 DEBUG:teuthology.parallel:result is None 2024-09-11T08:03:28.544 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T08:03:28.544 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T08:03:28.545 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-09-11T08:03:28.545 INFO:teuthology.orchestra.run.smithi196.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T08:03:28.545 INFO:teuthology.orchestra.run.smithi196.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T08:03:28.545 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T08:03:28.545 INFO:teuthology.orchestra.run.smithi196.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-11T08:03:28.545 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T08:03:28.545 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-11T08:03:28.943 DEBUG:teuthology.parallel:result is None 2024-09-11T08:03:28.943 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-09-11T08:03:28.956 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-09-11T08:03:28.956 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T08:03:28.956 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T08:03:28.976 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T08:03:28.976 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-09-11T08:03:29.034 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T08:03:29.035 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:03:29.035 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 975 Links: 1 2024-09-11T08:03:29.035 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:03:29.035 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:03:29.035 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-11 08:03:26.819286263 +0000 2024-09-11T08:03:29.035 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-11 08:03:26.510247845 +0000 2024-09-11T08:03:29.035 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-11 08:03:26.510247845 +0000 2024-09-11T08:03:29.035 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-11 08:03:26.510247845 +0000 2024-09-11T08:03:29.035 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T08:03:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-11T08:03:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-11T08:03:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000198413 s, 2.6 MB/s 2024-09-11T08:03:29.106 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T08:03:29.165 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-09-11T08:03:29.224 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T08:03:29.225 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:03:29.225 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 968 Links: 1 2024-09-11T08:03:29.225 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:03:29.225 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:03:29.225 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-11 08:03:26.819286263 +0000 2024-09-11T08:03:29.225 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-11 08:03:26.510247845 +0000 2024-09-11T08:03:29.225 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-11 08:03:26.510247845 +0000 2024-09-11T08:03:29.225 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-11 08:03:26.510247845 +0000 2024-09-11T08:03:29.225 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T08:03:29.291 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-11T08:03:29.291 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-11T08:03:29.291 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000219531 s, 2.3 MB/s 2024-09-11T08:03:29.293 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T08:03:29.350 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-09-11T08:03:29.407 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T08:03:29.408 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:03:29.408 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 979 Links: 1 2024-09-11T08:03:29.408 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:03:29.408 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:03:29.408 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-11 08:03:26.820286388 +0000 2024-09-11T08:03:29.408 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-11 08:03:26.510247845 +0000 2024-09-11T08:03:29.408 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-11 08:03:26.510247845 +0000 2024-09-11T08:03:29.408 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-11 08:03:26.510247845 +0000 2024-09-11T08:03:29.408 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T08:03:29.473 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-11T08:03:29.473 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-11T08:03:29.473 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000194243 s, 2.6 MB/s 2024-09-11T08:03:29.474 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T08:03:29.532 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-09-11T08:03:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T08:03:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:03:29.588 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-11T08:03:29.588 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:03:29.588 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:03:29.588 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-11 08:03:26.820286388 +0000 2024-09-11T08:03:29.588 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-11 08:03:26.510247845 +0000 2024-09-11T08:03:29.588 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-11 08:03:26.510247845 +0000 2024-09-11T08:03:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-11 08:03:26.510247845 +0000 2024-09-11T08:03:29.588 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T08:03:29.652 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-11T08:03:29.652 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-11T08:03:29.652 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000163818 s, 3.1 MB/s 2024-09-11T08:03:29.653 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T08:03:29.713 DEBUG:teuthology.orchestra.run.smithi071:> sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-11T08:03:29.858 INFO:teuthology.orchestra.run.smithi071.stdout:loop 2024-09-11T08:03:29.859 INFO:tasks.nvme_loop:Connecting nvme_loop smithi071:/dev/vg_nvme/lv_1... 2024-09-11T08:03:29.859 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-11T08:03:29.892 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-11T08:03:29.923 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/vg_nvme/lv_1 2024-09-11T08:03:29.933 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-11T08:03:29.969 INFO:teuthology.orchestra.run.smithi071.stdout:connecting to device: nvme1 2024-09-11T08:03:29.971 INFO:tasks.nvme_loop:Connecting nvme_loop smithi071:/dev/vg_nvme/lv_2... 2024-09-11T08:03:29.971 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-11T08:03:30.042 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-11T08:03:30.063 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/vg_nvme/lv_2 2024-09-11T08:03:30.073 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-11T08:03:30.108 INFO:teuthology.orchestra.run.smithi071.stdout:connecting to device: nvme2 2024-09-11T08:03:30.110 INFO:tasks.nvme_loop:Connecting nvme_loop smithi071:/dev/vg_nvme/lv_3... 2024-09-11T08:03:30.110 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-11T08:03:30.184 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-11T08:03:30.209 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/vg_nvme/lv_3 2024-09-11T08:03:30.219 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-11T08:03:30.249 INFO:teuthology.orchestra.run.smithi071.stdout:connecting to device: nvme3 2024-09-11T08:03:30.250 INFO:tasks.nvme_loop:Connecting nvme_loop smithi071:/dev/vg_nvme/lv_4... 2024-09-11T08:03:30.251 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-11T08:03:30.325 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-11T08:03:30.346 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/vg_nvme/lv_4 2024-09-11T08:03:30.356 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-11T08:03:30.406 INFO:teuthology.orchestra.run.smithi071.stdout:connecting to device: nvme4 2024-09-11T08:03:30.408 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T08:03:30.408 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T08:03:30.463 DEBUG:teuthology.orchestra.run.smithi071:> sudo nvme list -o json 2024-09-11T08:03:30.529 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T08:03:30.529 INFO:teuthology.orchestra.run.smithi071.stdout: "Devices":[ 2024-09-11T08:03:30.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T08:03:30.529 INFO:teuthology.orchestra.run.smithi071.stdout: "NameSpace":1, 2024-09-11T08:03:30.530 INFO:teuthology.orchestra.run.smithi071.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-11T08:03:30.530 INFO:teuthology.orchestra.run.smithi071.stdout: "GenericPath":"/dev/ng4n1", 2024-09-11T08:03:30.530 INFO:teuthology.orchestra.run.smithi071.stdout: "Firmware":"5.14.0-5", 2024-09-11T08:03:30.530 INFO:teuthology.orchestra.run.smithi071.stdout: "ModelNumber":"Linux", 2024-09-11T08:03:30.530 INFO:teuthology.orchestra.run.smithi071.stdout: "SerialNumber":"af01dfb17c76d89c6286", 2024-09-11T08:03:30.530 INFO:teuthology.orchestra.run.smithi071.stdout: "UsedBytes":95995035648, 2024-09-11T08:03:30.530 INFO:teuthology.orchestra.run.smithi071.stdout: "MaximumLBA":187490304, 2024-09-11T08:03:30.530 INFO:teuthology.orchestra.run.smithi071.stdout: "PhysicalSize":95995035648, 2024-09-11T08:03:30.530 INFO:teuthology.orchestra.run.smithi071.stdout: "SectorSize":512 2024-09-11T08:03:30.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T08:03:30.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T08:03:30.530 INFO:teuthology.orchestra.run.smithi071.stdout: "NameSpace":1, 2024-09-11T08:03:30.530 INFO:teuthology.orchestra.run.smithi071.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-11T08:03:30.531 INFO:teuthology.orchestra.run.smithi071.stdout: "GenericPath":"/dev/ng3n1", 2024-09-11T08:03:30.531 INFO:teuthology.orchestra.run.smithi071.stdout: "Firmware":"5.14.0-5", 2024-09-11T08:03:30.531 INFO:teuthology.orchestra.run.smithi071.stdout: "ModelNumber":"Linux", 2024-09-11T08:03:30.531 INFO:teuthology.orchestra.run.smithi071.stdout: "SerialNumber":"bd57c9dc53e965b94ed7", 2024-09-11T08:03:30.531 INFO:teuthology.orchestra.run.smithi071.stdout: "UsedBytes":95995035648, 2024-09-11T08:03:30.531 INFO:teuthology.orchestra.run.smithi071.stdout: "MaximumLBA":187490304, 2024-09-11T08:03:30.531 INFO:teuthology.orchestra.run.smithi071.stdout: "PhysicalSize":95995035648, 2024-09-11T08:03:30.531 INFO:teuthology.orchestra.run.smithi071.stdout: "SectorSize":512 2024-09-11T08:03:30.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T08:03:30.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T08:03:30.531 INFO:teuthology.orchestra.run.smithi071.stdout: "NameSpace":1, 2024-09-11T08:03:30.531 INFO:teuthology.orchestra.run.smithi071.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-11T08:03:30.531 INFO:teuthology.orchestra.run.smithi071.stdout: "GenericPath":"/dev/ng2n1", 2024-09-11T08:03:30.531 INFO:teuthology.orchestra.run.smithi071.stdout: "Firmware":"5.14.0-5", 2024-09-11T08:03:30.531 INFO:teuthology.orchestra.run.smithi071.stdout: "ModelNumber":"Linux", 2024-09-11T08:03:30.532 INFO:teuthology.orchestra.run.smithi071.stdout: "SerialNumber":"ad37d5b30696ccfca3ce", 2024-09-11T08:03:30.532 INFO:teuthology.orchestra.run.smithi071.stdout: "UsedBytes":95995035648, 2024-09-11T08:03:30.532 INFO:teuthology.orchestra.run.smithi071.stdout: "MaximumLBA":187490304, 2024-09-11T08:03:30.532 INFO:teuthology.orchestra.run.smithi071.stdout: "PhysicalSize":95995035648, 2024-09-11T08:03:30.532 INFO:teuthology.orchestra.run.smithi071.stdout: "SectorSize":512 2024-09-11T08:03:30.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T08:03:30.532 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T08:03:30.532 INFO:teuthology.orchestra.run.smithi071.stdout: "NameSpace":1, 2024-09-11T08:03:30.532 INFO:teuthology.orchestra.run.smithi071.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-11T08:03:30.532 INFO:teuthology.orchestra.run.smithi071.stdout: "GenericPath":"/dev/ng1n1", 2024-09-11T08:03:30.532 INFO:teuthology.orchestra.run.smithi071.stdout: "Firmware":"5.14.0-5", 2024-09-11T08:03:30.532 INFO:teuthology.orchestra.run.smithi071.stdout: "ModelNumber":"Linux", 2024-09-11T08:03:30.532 INFO:teuthology.orchestra.run.smithi071.stdout: "SerialNumber":"e546ccad77aaa03a064c", 2024-09-11T08:03:30.532 INFO:teuthology.orchestra.run.smithi071.stdout: "UsedBytes":95995035648, 2024-09-11T08:03:30.533 INFO:teuthology.orchestra.run.smithi071.stdout: "MaximumLBA":187490304, 2024-09-11T08:03:30.533 INFO:teuthology.orchestra.run.smithi071.stdout: "PhysicalSize":95995035648, 2024-09-11T08:03:30.533 INFO:teuthology.orchestra.run.smithi071.stdout: "SectorSize":512 2024-09-11T08:03:30.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-11T08:03:30.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-11T08:03:30.533 INFO:teuthology.orchestra.run.smithi071.stdout: "NameSpace":1, 2024-09-11T08:03:30.533 INFO:teuthology.orchestra.run.smithi071.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-11T08:03:30.533 INFO:teuthology.orchestra.run.smithi071.stdout: "GenericPath":"/dev/ng0n1", 2024-09-11T08:03:30.533 INFO:teuthology.orchestra.run.smithi071.stdout: "Firmware":"8DV101H0", 2024-09-11T08:03:30.533 INFO:teuthology.orchestra.run.smithi071.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-11T08:03:30.533 INFO:teuthology.orchestra.run.smithi071.stdout: "SerialNumber":"PHFT62040172400BGN", 2024-09-11T08:03:30.533 INFO:teuthology.orchestra.run.smithi071.stdout: "UsedBytes":400088457216, 2024-09-11T08:03:30.533 INFO:teuthology.orchestra.run.smithi071.stdout: "MaximumLBA":781422768, 2024-09-11T08:03:30.533 INFO:teuthology.orchestra.run.smithi071.stdout: "PhysicalSize":400088457216, 2024-09-11T08:03:30.533 INFO:teuthology.orchestra.run.smithi071.stdout: "SectorSize":512 2024-09-11T08:03:30.534 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-11T08:03:30.534 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T08:03:30.534 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T08:03:30.534 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-11T08:03:30.535 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T08:03:30.535 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/scratch_devs 2024-09-11T08:03:30.600 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T08:03:30.600 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T08:03:30.620 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T08:03:30.620 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-09-11T08:03:30.679 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T08:03:30.679 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:03:30.679 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 956 Links: 1 2024-09-11T08:03:30.679 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:03:30.679 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:03:30.679 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-11 08:03:27.481360015 +0000 2024-09-11T08:03:30.679 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-11 08:03:27.146368023 +0000 2024-09-11T08:03:30.679 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-11 08:03:27.146368023 +0000 2024-09-11T08:03:30.679 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-11 08:03:27.146368023 +0000 2024-09-11T08:03:30.679 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T08:03:30.746 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-11T08:03:30.746 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-11T08:03:30.746 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.00028969 s, 1.8 MB/s 2024-09-11T08:03:30.748 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T08:03:30.806 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-09-11T08:03:30.863 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T08:03:30.863 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:03:30.863 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 981 Links: 1 2024-09-11T08:03:30.863 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:03:30.863 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:03:30.863 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-11 08:03:27.481360015 +0000 2024-09-11T08:03:30.863 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-11 08:03:27.149367929 +0000 2024-09-11T08:03:30.863 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-11 08:03:27.149367929 +0000 2024-09-11T08:03:30.863 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-11 08:03:27.149367929 +0000 2024-09-11T08:03:30.863 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T08:03:30.928 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-11T08:03:30.928 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-11T08:03:30.928 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000202182 s, 2.5 MB/s 2024-09-11T08:03:30.930 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T08:03:30.987 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-09-11T08:03:31.043 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T08:03:31.043 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:03:31.043 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 979 Links: 1 2024-09-11T08:03:31.043 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:03:31.043 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:03:31.043 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-11 08:03:27.482359993 +0000 2024-09-11T08:03:31.043 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-11 08:03:27.149367929 +0000 2024-09-11T08:03:31.044 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-11 08:03:27.149367929 +0000 2024-09-11T08:03:31.044 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-11 08:03:27.149367929 +0000 2024-09-11T08:03:31.044 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T08:03:31.107 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-11T08:03:31.107 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-11T08:03:31.107 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000214193 s, 2.4 MB/s 2024-09-11T08:03:31.108 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T08:03:31.165 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-09-11T08:03:31.221 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T08:03:31.221 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:03:31.222 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-11T08:03:31.222 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:03:31.222 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:03:31.222 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-11 08:03:27.482359993 +0000 2024-09-11T08:03:31.222 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-11 08:03:27.148367960 +0000 2024-09-11T08:03:31.222 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-11 08:03:27.148367960 +0000 2024-09-11T08:03:31.222 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-11 08:03:27.148367960 +0000 2024-09-11T08:03:31.222 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T08:03:31.285 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-11T08:03:31.285 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-11T08:03:31.285 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000205239 s, 2.5 MB/s 2024-09-11T08:03:31.286 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T08:03:31.343 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-11T08:03:31.477 INFO:teuthology.orchestra.run.smithi196.stdout:loop 2024-09-11T08:03:31.479 INFO:tasks.nvme_loop:Connecting nvme_loop smithi196:/dev/vg_nvme/lv_1... 2024-09-11T08:03:31.479 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-11T08:03:31.511 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-11T08:03:31.531 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/vg_nvme/lv_1 2024-09-11T08:03:31.541 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-11T08:03:31.576 INFO:teuthology.orchestra.run.smithi196.stdout:connecting to device: nvme1 2024-09-11T08:03:31.577 INFO:tasks.nvme_loop:Connecting nvme_loop smithi196:/dev/vg_nvme/lv_2... 2024-09-11T08:03:31.578 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-11T08:03:31.652 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-11T08:03:31.674 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/vg_nvme/lv_2 2024-09-11T08:03:31.685 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-11T08:03:31.715 INFO:teuthology.orchestra.run.smithi196.stdout:connecting to device: nvme2 2024-09-11T08:03:31.717 INFO:tasks.nvme_loop:Connecting nvme_loop smithi196:/dev/vg_nvme/lv_3... 2024-09-11T08:03:31.718 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-11T08:03:31.789 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-11T08:03:31.810 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/vg_nvme/lv_3 2024-09-11T08:03:31.820 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-11T08:03:31.855 INFO:teuthology.orchestra.run.smithi196.stdout:connecting to device: nvme3 2024-09-11T08:03:31.857 INFO:tasks.nvme_loop:Connecting nvme_loop smithi196:/dev/vg_nvme/lv_4... 2024-09-11T08:03:31.857 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-11T08:03:31.931 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-11T08:03:31.952 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/vg_nvme/lv_4 2024-09-11T08:03:31.963 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-11T08:03:31.998 INFO:teuthology.orchestra.run.smithi196.stdout:connecting to device: nvme4 2024-09-11T08:03:32.000 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T08:03:32.000 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T08:03:32.056 DEBUG:teuthology.orchestra.run.smithi196:> sudo nvme list -o json 2024-09-11T08:03:32.123 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-11T08:03:32.123 INFO:teuthology.orchestra.run.smithi196.stdout: "Devices":[ 2024-09-11T08:03:32.123 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-09-11T08:03:32.123 INFO:teuthology.orchestra.run.smithi196.stdout: "NameSpace":1, 2024-09-11T08:03:32.123 INFO:teuthology.orchestra.run.smithi196.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-11T08:03:32.123 INFO:teuthology.orchestra.run.smithi196.stdout: "GenericPath":"/dev/ng4n1", 2024-09-11T08:03:32.123 INFO:teuthology.orchestra.run.smithi196.stdout: "Firmware":"5.14.0-5", 2024-09-11T08:03:32.123 INFO:teuthology.orchestra.run.smithi196.stdout: "ModelNumber":"Linux", 2024-09-11T08:03:32.123 INFO:teuthology.orchestra.run.smithi196.stdout: "SerialNumber":"c718f8db9a8aac448b93", 2024-09-11T08:03:32.123 INFO:teuthology.orchestra.run.smithi196.stdout: "UsedBytes":95995035648, 2024-09-11T08:03:32.123 INFO:teuthology.orchestra.run.smithi196.stdout: "MaximumLBA":187490304, 2024-09-11T08:03:32.124 INFO:teuthology.orchestra.run.smithi196.stdout: "PhysicalSize":95995035648, 2024-09-11T08:03:32.124 INFO:teuthology.orchestra.run.smithi196.stdout: "SectorSize":512 2024-09-11T08:03:32.124 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-11T08:03:32.124 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-09-11T08:03:32.124 INFO:teuthology.orchestra.run.smithi196.stdout: "NameSpace":1, 2024-09-11T08:03:32.124 INFO:teuthology.orchestra.run.smithi196.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-11T08:03:32.124 INFO:teuthology.orchestra.run.smithi196.stdout: "GenericPath":"/dev/ng3n1", 2024-09-11T08:03:32.124 INFO:teuthology.orchestra.run.smithi196.stdout: "Firmware":"5.14.0-5", 2024-09-11T08:03:32.124 INFO:teuthology.orchestra.run.smithi196.stdout: "ModelNumber":"Linux", 2024-09-11T08:03:32.124 INFO:teuthology.orchestra.run.smithi196.stdout: "SerialNumber":"ee8c8ccdac0431c94aaf", 2024-09-11T08:03:32.124 INFO:teuthology.orchestra.run.smithi196.stdout: "UsedBytes":95995035648, 2024-09-11T08:03:32.124 INFO:teuthology.orchestra.run.smithi196.stdout: "MaximumLBA":187490304, 2024-09-11T08:03:32.124 INFO:teuthology.orchestra.run.smithi196.stdout: "PhysicalSize":95995035648, 2024-09-11T08:03:32.125 INFO:teuthology.orchestra.run.smithi196.stdout: "SectorSize":512 2024-09-11T08:03:32.125 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-11T08:03:32.125 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-09-11T08:03:32.125 INFO:teuthology.orchestra.run.smithi196.stdout: "NameSpace":1, 2024-09-11T08:03:32.125 INFO:teuthology.orchestra.run.smithi196.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-11T08:03:32.125 INFO:teuthology.orchestra.run.smithi196.stdout: "GenericPath":"/dev/ng2n1", 2024-09-11T08:03:32.125 INFO:teuthology.orchestra.run.smithi196.stdout: "Firmware":"5.14.0-5", 2024-09-11T08:03:32.125 INFO:teuthology.orchestra.run.smithi196.stdout: "ModelNumber":"Linux", 2024-09-11T08:03:32.125 INFO:teuthology.orchestra.run.smithi196.stdout: "SerialNumber":"767a095da268aa803e9d", 2024-09-11T08:03:32.125 INFO:teuthology.orchestra.run.smithi196.stdout: "UsedBytes":95995035648, 2024-09-11T08:03:32.125 INFO:teuthology.orchestra.run.smithi196.stdout: "MaximumLBA":187490304, 2024-09-11T08:03:32.125 INFO:teuthology.orchestra.run.smithi196.stdout: "PhysicalSize":95995035648, 2024-09-11T08:03:32.126 INFO:teuthology.orchestra.run.smithi196.stdout: "SectorSize":512 2024-09-11T08:03:32.126 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-11T08:03:32.126 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-09-11T08:03:32.126 INFO:teuthology.orchestra.run.smithi196.stdout: "NameSpace":1, 2024-09-11T08:03:32.126 INFO:teuthology.orchestra.run.smithi196.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-11T08:03:32.126 INFO:teuthology.orchestra.run.smithi196.stdout: "GenericPath":"/dev/ng1n1", 2024-09-11T08:03:32.126 INFO:teuthology.orchestra.run.smithi196.stdout: "Firmware":"5.14.0-5", 2024-09-11T08:03:32.126 INFO:teuthology.orchestra.run.smithi196.stdout: "ModelNumber":"Linux", 2024-09-11T08:03:32.126 INFO:teuthology.orchestra.run.smithi196.stdout: "SerialNumber":"d2b7d9f4cf5553bccfc7", 2024-09-11T08:03:32.126 INFO:teuthology.orchestra.run.smithi196.stdout: "UsedBytes":95995035648, 2024-09-11T08:03:32.126 INFO:teuthology.orchestra.run.smithi196.stdout: "MaximumLBA":187490304, 2024-09-11T08:03:32.126 INFO:teuthology.orchestra.run.smithi196.stdout: "PhysicalSize":95995035648, 2024-09-11T08:03:32.127 INFO:teuthology.orchestra.run.smithi196.stdout: "SectorSize":512 2024-09-11T08:03:32.127 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-11T08:03:32.127 INFO:teuthology.orchestra.run.smithi196.stdout: { 2024-09-11T08:03:32.127 INFO:teuthology.orchestra.run.smithi196.stdout: "NameSpace":1, 2024-09-11T08:03:32.127 INFO:teuthology.orchestra.run.smithi196.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-11T08:03:32.127 INFO:teuthology.orchestra.run.smithi196.stdout: "GenericPath":"/dev/ng0n1", 2024-09-11T08:03:32.127 INFO:teuthology.orchestra.run.smithi196.stdout: "Firmware":"8DV101H0", 2024-09-11T08:03:32.127 INFO:teuthology.orchestra.run.smithi196.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-11T08:03:32.127 INFO:teuthology.orchestra.run.smithi196.stdout: "SerialNumber":"CVFT6246000Z400BGN", 2024-09-11T08:03:32.127 INFO:teuthology.orchestra.run.smithi196.stdout: "UsedBytes":400088457216, 2024-09-11T08:03:32.127 INFO:teuthology.orchestra.run.smithi196.stdout: "MaximumLBA":781422768, 2024-09-11T08:03:32.128 INFO:teuthology.orchestra.run.smithi196.stdout: "PhysicalSize":400088457216, 2024-09-11T08:03:32.128 INFO:teuthology.orchestra.run.smithi196.stdout: "SectorSize":512 2024-09-11T08:03:32.128 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-11T08:03:32.128 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-09-11T08:03:32.128 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-11T08:03:32.129 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-11T08:03:32.129 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T08:03:32.129 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/scratch_devs 2024-09-11T08:03:32.194 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T08:03:32.277 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': False}, 'global': {'mon election default strategy': 3}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'MON_DOWN', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'} 2024-09-11T08:03:32.278 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:03:32.278 INFO:tasks.cephadm:Cluster fsid is 566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:03:32.278 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T08:03:32.279 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.71', 'mon.c': '[v2:172.21.15.71:3301,v1:172.21.15.71:6790]', 'mon.b': '172.21.15.196'} 2024-09-11T08:03:32.279 INFO:tasks.cephadm:First mon is mon.a on smithi071 2024-09-11T08:03:32.279 INFO:tasks.cephadm:First mgr is y 2024-09-11T08:03:32.279 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T08:03:32.279 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname $(hostname -s) 2024-09-11T08:03:32.311 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-09-11T08:03:32.336 INFO:tasks.cephadm:Downloading cephadm (repo https://git.ceph.com/ceph-ci.git ref d0e6828a2016d48cf25ad84064e50742bb1c39b9)... 2024-09-11T08:03:32.336 DEBUG:teuthology.orchestra.run.smithi071:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show d0e6828a2016d48cf25ad84064e50742bb1c39b9:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T08:03:32.370 INFO:teuthology.orchestra.run.smithi071.stderr:Cloning into 'testrepo'... 2024-09-11T08:04:31.046 INFO:teuthology.orchestra.run.smithi071.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 11 08:04 /home/ubuntu/cephtest/cephadm 2024-09-11T08:04:31.047 DEBUG:teuthology.orchestra.run.smithi196:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show d0e6828a2016d48cf25ad84064e50742bb1c39b9:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T08:04:31.067 INFO:teuthology.orchestra.run.smithi196.stderr:Cloning into 'testrepo'... 2024-09-11T08:05:28.313 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 11 08:05 /home/ubuntu/cephtest/cephadm 2024-09-11T08:05:28.314 DEBUG:teuthology.orchestra.run.smithi071:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T08:05:28.335 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T08:05:28.382 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 on all hosts... 2024-09-11T08:05:28.383 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T08:05:28.385 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T08:05:30.040 INFO:teuthology.orchestra.run.smithi071.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:05:31.736 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:05:58.198 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T08:05:58.198 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:05:58.198 INFO:teuthology.orchestra.run.smithi071.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T08:05:58.198 INFO:teuthology.orchestra.run.smithi071.stdout: "repo_digests": [ 2024-09-11T08:05:58.198 INFO:teuthology.orchestra.run.smithi071.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T08:05:58.198 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T08:05:58.198 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T08:06:00.206 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-11T08:06:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:06:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T08:06:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-09-11T08:06:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T08:06:00.206 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-09-11T08:06:00.206 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-11T08:06:00.226 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph 2024-09-11T08:06:00.260 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-09-11T08:06:00.289 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /etc/ceph 2024-09-11T08:06:00.328 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-09-11T08:06:00.354 INFO:tasks.cephadm:Writing seed config... 2024-09-11T08:06:00.355 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T08:06:00.356 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T08:06:00.356 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = False 2024-09-11T08:06:00.356 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-09-11T08:06:00.356 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T08:06:00.356 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T08:06:00.356 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T08:06:00.356 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T08:06:00.356 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T08:06:00.356 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T08:06:00.357 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T08:06:00.357 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T08:06:00.385 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 566367e6-7014-11ef-bcea-c7b262605968 mon election default strategy = 3 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = False [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-11T08:06:00.385 DEBUG:teuthology.orchestra.run.smithi071:mon.a> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T08:06:00.428 DEBUG:teuthology.orchestra.run.smithi071:mgr.y> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T08:06:00.471 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T08:06:00.471 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 -v bootstrap --fsid 566367e6-7014-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.71 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:06:00.668 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:06:00.668 INFO:teuthology.orchestra.run.smithi071.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9', '-v', 'bootstrap', '--fsid', '566367e6-7014-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.71', '--skip-admin-label'] 2024-09-11T08:06:00.698 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 5.2.2 2024-09-11T08:06:00.699 INFO:teuthology.orchestra.run.smithi071.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T08:06:00.699 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying podman|docker is present... 2024-09-11T08:06:00.722 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 5.2.2 2024-09-11T08:06:00.722 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying lvm2 is present... 2024-09-11T08:06:00.722 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying time synchronization is in place... 2024-09-11T08:06:00.729 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T08:06:00.730 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T08:06:00.736 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T08:06:00.736 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-11T08:06:00.743 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout enabled 2024-09-11T08:06:00.749 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout active 2024-09-11T08:06:00.749 INFO:teuthology.orchestra.run.smithi071.stdout:Unit chronyd.service is enabled and running 2024-09-11T08:06:00.749 INFO:teuthology.orchestra.run.smithi071.stdout:Repeating the final host check... 2024-09-11T08:06:00.771 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 5.2.2 2024-09-11T08:06:00.771 INFO:teuthology.orchestra.run.smithi071.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T08:06:00.771 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl is present 2024-09-11T08:06:00.771 INFO:teuthology.orchestra.run.smithi071.stdout:lvcreate is present 2024-09-11T08:06:00.778 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T08:06:00.778 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T08:06:00.784 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T08:06:00.784 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-11T08:06:00.791 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout enabled 2024-09-11T08:06:00.798 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout active 2024-09-11T08:06:00.798 INFO:teuthology.orchestra.run.smithi071.stdout:Unit chronyd.service is enabled and running 2024-09-11T08:06:00.798 INFO:teuthology.orchestra.run.smithi071.stdout:Host looks OK 2024-09-11T08:06:00.798 INFO:teuthology.orchestra.run.smithi071.stdout:Cluster fsid: 566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:06:00.798 INFO:teuthology.orchestra.run.smithi071.stdout:Acquiring lock 139800439513776 on /run/cephadm/566367e6-7014-11ef-bcea-c7b262605968.lock 2024-09-11T08:06:00.798 INFO:teuthology.orchestra.run.smithi071.stdout:Lock 139800439513776 acquired on /run/cephadm/566367e6-7014-11ef-bcea-c7b262605968.lock 2024-09-11T08:06:00.799 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying IP 172.21.15.71 port 3300 ... 2024-09-11T08:06:00.799 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying IP 172.21.15.71 port 6789 ... 2024-09-11T08:06:00.799 INFO:teuthology.orchestra.run.smithi071.stdout:Base mon IP(s) is [172.21.15.71:3300, 172.21.15.71:6789], mon addrv is [v2:172.21.15.71:3300,v1:172.21.15.71:6789] 2024-09-11T08:06:00.802 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.71 metric 100 2024-09-11T08:06:00.802 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.71 metric 100 2024-09-11T08:06:00.805 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T08:06:00.805 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-11T08:06:00.805 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-11T08:06:00.808 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T08:06:00.808 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T08:06:00.808 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T08:06:00.808 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-11T08:06:00.808 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d765/64 scope link noprefixroute 2024-09-11T08:06:00.808 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T08:06:00.809 INFO:teuthology.orchestra.run.smithi071.stdout:Mon IP `172.21.15.71` is in CIDR network `172.21.0.0/20` 2024-09-11T08:06:00.809 INFO:teuthology.orchestra.run.smithi071.stdout:Mon IP `172.21.15.71` is in CIDR network `172.21.0.0/20` 2024-09-11T08:06:00.809 INFO:teuthology.orchestra.run.smithi071.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-11T08:06:00.809 INFO:teuthology.orchestra.run.smithi071.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T08:06:00.810 INFO:teuthology.orchestra.run.smithi071.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:06:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T08:06:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:06:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T08:06:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Copying blob sha256:0d2f264d63cec2f6fb0f85eed7b2c8c5b4db3ab9edd9d71a74c04647294a553f 2024-09-11T08:06:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T08:06:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Copying config sha256:4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T08:06:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T08:06:02.093 INFO:teuthology.orchestra.run.smithi071.stdout:ceph: stdout ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T08:06:02.093 INFO:teuthology.orchestra.run.smithi071.stdout:Ceph version: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T08:06:02.094 INFO:teuthology.orchestra.run.smithi071.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T08:06:03.106 INFO:teuthology.orchestra.run.smithi071.stdout:stat: stdout 167 167 2024-09-11T08:06:03.106 INFO:teuthology.orchestra.run.smithi071.stdout:Creating initial keys... 2024-09-11T08:06:04.011 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph-authtool: stdout AQBrT+FmqAaPGRAAz42SQ0ec286O31KGVF7KvQ== 2024-09-11T08:06:04.991 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph-authtool: stdout AQBsT+FmCd5eFRAAoD33ObVwdtBKMi9L/4Fe6w== 2024-09-11T08:06:05.896 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph-authtool: stdout AQBtT+FmPvAzEhAA87v+Cx8euYorukLaLqVxwg== 2024-09-11T08:06:05.897 INFO:teuthology.orchestra.run.smithi071.stdout:Creating initial monmap... 2024-09-11T08:06:06.721 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T08:06:06.721 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-11T08:06:06.721 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:06:06.721 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T08:06:06.721 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool for a [v2:172.21.15.71:3300,v1:172.21.15.71:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T08:06:06.721 INFO:teuthology.orchestra.run.smithi071.stdout:setting min_mon_release = octopus 2024-09-11T08:06:06.721 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: set fsid to 566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:06:06.721 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T08:06:06.722 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:06:06.722 INFO:teuthology.orchestra.run.smithi071.stdout:Creating mon... 2024-09-11T08:06:07.649 INFO:teuthology.orchestra.run.smithi071.stdout:create mon.a on 2024-09-11T08:06:07.996 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T08:06:08.148 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-566367e6-7014-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-566367e6-7014-11ef-bcea-c7b262605968.target. 2024-09-11T08:06:08.149 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-566367e6-7014-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-566367e6-7014-11ef-bcea-c7b262605968.target. 2024-09-11T08:06:08.320 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-566367e6-7014-11ef-bcea-c7b262605968@mon.a 2024-09-11T08:06:08.320 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to reset failed state of unit ceph-566367e6-7014-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-566367e6-7014-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T08:06:08.486 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-566367e6-7014-11ef-bcea-c7b262605968.target.wants/ceph-566367e6-7014-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-566367e6-7014-11ef-bcea-c7b262605968@.service. 2024-09-11T08:06:08.957 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:08 smithi071 ceph-mon[25358]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T08:06:09.013 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:06:09.013 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-09-11T08:06:09.020 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:06:09.020 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-11T08:06:09.020 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-09-11T08:06:09.020 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T08:06:09.020 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mon to start... 2024-09-11T08:06:09.020 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mon... 2024-09-11T08:06:09.243 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:08 smithi071 bash[25320]: 63c08fc3261d06926ab07494e9b2f8670e124d7ff35a2c597b52fe22aa88eea8 2024-09-11T08:06:09.243 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:09 smithi071 systemd[1]: Started Ceph mon.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:06:10.066 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:09 smithi071 ceph-mon[25358]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T08:06:10.066 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:09 smithi071 ceph-mon[25358]: monmap e1: 1 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:06:10.066 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:09 smithi071 ceph-mon[25358]: fsmap 2024-09-11T08:06:10.066 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:09 smithi071 ceph-mon[25358]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T08:06:10.067 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:09 smithi071 ceph-mon[25358]: mgrmap e1: no daemons active 2024-09-11T08:06:10.067 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:09 smithi071 ceph-mon[25358]: from='client.? 172.21.15.71:0/1111245897' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T08:06:10.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T08:06:10.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout id: 566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:06:10.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T08:06:10.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T08:06:10.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout services: 2024-09-11T08:06:10.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.572687s) 2024-09-11T08:06:10.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T08:06:10.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T08:06:10.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T08:06:10.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout data: 2024-09-11T08:06:10.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T08:06:10.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T08:06:10.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T08:06:10.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T08:06:10.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T08:06:10.069 INFO:teuthology.orchestra.run.smithi071.stdout:mon is available 2024-09-11T08:06:10.070 INFO:teuthology.orchestra.run.smithi071.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T08:06:11.082 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T08:06:11.082 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T08:06:11.082 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout fsid = 566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:06:11.082 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T08:06:11.083 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.71:3300,v1:172.21.15.71:6789] 2024-09-11T08:06:11.083 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T08:06:11.083 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T08:06:11.083 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T08:06:11.083 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T08:06:11.083 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T08:06:11.083 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T08:06:11.083 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = False 2024-09-11T08:06:11.084 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T08:06:11.084 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T08:06:11.084 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T08:06:11.084 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T08:06:11.084 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T08:06:11.084 INFO:teuthology.orchestra.run.smithi071.stdout:Generating new minimal ceph.conf... 2024-09-11T08:06:11.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:10 smithi071 ceph-mon[25358]: from='client.? 172.21.15.71:0/398094218' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T08:06:11.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:10 smithi071 ceph-mon[25358]: from='client.? 172.21.15.71:0/398094218' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T08:06:12.143 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:11 smithi071 ceph-mon[25358]: from='client.? 172.21.15.71:0/1684996245' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:12.144 INFO:teuthology.orchestra.run.smithi071.stdout:Restarting the monitor... 2024-09-11T08:06:12.406 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:12 smithi071 systemd[1]: Stopping Ceph mon.a for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:06:12.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:12 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-a[25335]: 2024-09-11T08:06:12.404+0000 7f6dd3009640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:06:12.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:12 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-a[25335]: 2024-09-11T08:06:12.404+0000 7f6dd3009640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T08:06:12.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:12 smithi071 podman[25678]: 2024-09-11 08:06:12.466101547 +0000 UTC m=+0.151634980 container died 63c08fc3261d06926ab07494e9b2f8670e124d7ff35a2c597b52fe22aa88eea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T08:06:12.995 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:12 smithi071 podman[25678]: 2024-09-11 08:06:12.810616222 +0000 UTC m=+0.496149656 container cleanup 63c08fc3261d06926ab07494e9b2f8670e124d7ff35a2c597b52fe22aa88eea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T08:06:12.996 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:12 smithi071 bash[25678]: ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-a 2024-09-11T08:06:13.404 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:12 smithi071 podman[25696]: 2024-09-11 08:06:12.994895676 +0000 UTC m=+0.523570921 container remove 63c08fc3261d06926ab07494e9b2f8670e124d7ff35a2c597b52fe22aa88eea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:06:13.404 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T08:06:13.404 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 systemd[1]: Stopped Ceph mon.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:06:13.404 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 systemd[1]: Starting Ceph mon.a for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:06:13.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 podman[25796]: 2024-09-11 08:06:13.403761123 +0000 UTC m=+0.100636928 container create e6731b5b742dd722fd8592e46902bb9992bc9df713c2cc77ea95078a242e7a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-11T08:06:13.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 podman[25796]: 2024-09-11 08:06:13.318641354 +0000 UTC m=+0.015517158 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:06:13.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 podman[25796]: 2024-09-11 08:06:13.581172484 +0000 UTC m=+0.278048289 container init e6731b5b742dd722fd8592e46902bb9992bc9df713c2cc77ea95078a242e7a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-11T08:06:13.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 podman[25796]: 2024-09-11 08:06:13.586132396 +0000 UTC m=+0.283008200 container start e6731b5b742dd722fd8592e46902bb9992bc9df713c2cc77ea95078a242e7a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:06:13.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T08:06:13.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 7 2024-09-11T08:06:13.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: pidfile_write: ignore empty --pid-file 2024-09-11T08:06:13.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: load: jerasure load: lrc 2024-09-11T08:06:13.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: RocksDB version: 6.15.5 2024-09-11T08:06:13.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T08:06:13.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Compile date Sep 10 2024 2024-09-11T08:06:13.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: DB SUMMARY 2024-09-11T08:06:13.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: DB Session ID: 1EX28L2JKN1ZS3VO3JT4 2024-09-11T08:06:13.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: CURRENT file: CURRENT 2024-09-11T08:06:13.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T08:06:13.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-11T08:06:13.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T08:06:13.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 82852 ; 2024-09-11T08:06:13.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.error_if_exists: 0 2024-09-11T08:06:13.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.create_if_missing: 0 2024-09-11T08:06:13.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.paranoid_checks: 1 2024-09-11T08:06:13.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T08:06:13.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.env: 0x5578eb7ad440 2024-09-11T08:06:13.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.fs: Posix File System 2024-09-11T08:06:13.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.info_log: 0x5578edf68ec0 2024-09-11T08:06:13.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T08:06:13.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.statistics: (nil) 2024-09-11T08:06:13.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.use_fsync: 0 2024-09-11T08:06:13.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_log_file_size: 0 2024-09-11T08:06:13.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T08:06:13.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T08:06:13.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T08:06:13.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T08:06:13.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.allow_fallocate: 1 2024-09-11T08:06:13.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T08:06:13.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T08:06:13.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.use_direct_reads: 0 2024-09-11T08:06:13.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T08:06:13.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T08:06:13.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.db_log_dir: 2024-09-11T08:06:13.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-11T08:06:13.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T08:06:13.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T08:06:13.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T08:06:13.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T08:06:13.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T08:06:13.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T08:06:13.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T08:06:13.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T08:06:13.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.write_buffer_manager: 0x5578ee0267e0 2024-09-11T08:06:13.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T08:06:13.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T08:06:13.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T08:06:13.676 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T08:06:13.676 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T08:06:13.676 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T08:06:13.677 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T08:06:13.677 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T08:06:13.677 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T08:06:13.677 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.unordered_write: 0 2024-09-11T08:06:13.677 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T08:06:13.677 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T08:06:13.677 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T08:06:13.677 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T08:06:13.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.row_cache: None 2024-09-11T08:06:13.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.wal_filter: None 2024-09-11T08:06:13.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T08:06:13.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T08:06:13.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.preserve_deletes: 0 2024-09-11T08:06:13.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.two_write_queues: 0 2024-09-11T08:06:13.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T08:06:13.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.atomic_flush: 0 2024-09-11T08:06:13.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T08:06:13.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T08:06:13.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T08:06:13.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.log_readahead_size: 0 2024-09-11T08:06:13.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T08:06:13.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T08:06:13.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T08:06:13.679 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T08:06:13.679 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T08:06:13.679 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T08:06:13.679 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_background_jobs: 2 2024-09-11T08:06:13.679 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_background_compactions: -1 2024-09-11T08:06:13.679 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_subcompactions: 1 2024-09-11T08:06:13.679 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T08:06:13.679 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T08:06:13.679 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T08:06:13.679 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T08:06:13.679 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T08:06:13.679 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T08:06:13.679 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T08:06:13.679 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T08:06:13.679 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_open_files: -1 2024-09-11T08:06:13.680 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T08:06:13.680 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T08:06:13.680 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T08:06:13.680 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T08:06:13.680 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_background_flushes: -1 2024-09-11T08:06:13.680 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Compression algorithms supported: 2024-09-11T08:06:13.680 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: kZSTD supported: 0 2024-09-11T08:06:13.680 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: kXpressCompression supported: 0 2024-09-11T08:06:13.680 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: kBZip2Compression supported: 0 2024-09-11T08:06:13.680 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T08:06:13.680 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: kLZ4Compression supported: 1 2024-09-11T08:06:13.680 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: kZlibCompression supported: 1 2024-09-11T08:06:13.680 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T08:06:13.681 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: kSnappyCompression supported: 1 2024-09-11T08:06:13.681 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T08:06:13.681 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-11T08:06:13.681 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T08:06:13.681 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T08:06:13.681 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.merge_operator: 2024-09-11T08:06:13.681 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compaction_filter: None 2024-09-11T08:06:13.681 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T08:06:13.681 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T08:06:13.682 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T08:06:13.682 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T08:06:13.682 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5578edf0fa38) 2024-09-11T08:06:13.682 INFO:journalctl@ceph.mon.a.smithi071.stdout: cache_index_and_filter_blocks: 1 2024-09-11T08:06:13.682 INFO:journalctl@ceph.mon.a.smithi071.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T08:06:13.682 INFO:journalctl@ceph.mon.a.smithi071.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T08:06:13.682 INFO:journalctl@ceph.mon.a.smithi071.stdout: pin_top_level_index_and_filter: 1 2024-09-11T08:06:13.682 INFO:journalctl@ceph.mon.a.smithi071.stdout: index_type: 0 2024-09-11T08:06:13.682 INFO:journalctl@ceph.mon.a.smithi071.stdout: data_block_index_type: 0 2024-09-11T08:06:13.682 INFO:journalctl@ceph.mon.a.smithi071.stdout: index_shortening: 1 2024-09-11T08:06:13.682 INFO:journalctl@ceph.mon.a.smithi071.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T08:06:13.682 INFO:journalctl@ceph.mon.a.smithi071.stdout: hash_index_allow_collision: 1 2024-09-11T08:06:13.682 INFO:journalctl@ceph.mon.a.smithi071.stdout: checksum: 1 2024-09-11T08:06:13.682 INFO:journalctl@ceph.mon.a.smithi071.stdout: no_block_cache: 0 2024-09-11T08:06:13.683 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_cache: 0x5578edf451f0 2024-09-11T08:06:13.683 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_cache_name: BinnedLRUCache 2024-09-11T08:06:13.683 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_cache_options: 2024-09-11T08:06:13.683 INFO:journalctl@ceph.mon.a.smithi071.stdout: capacity : 536870912 2024-09-11T08:06:13.683 INFO:journalctl@ceph.mon.a.smithi071.stdout: num_shard_bits : 4 2024-09-11T08:06:13.683 INFO:journalctl@ceph.mon.a.smithi071.stdout: strict_capacity_limit : 0 2024-09-11T08:06:13.683 INFO:journalctl@ceph.mon.a.smithi071.stdout: high_pri_pool_ratio: 0.000 2024-09-11T08:06:13.683 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_cache_compressed: (nil) 2024-09-11T08:06:13.683 INFO:journalctl@ceph.mon.a.smithi071.stdout: persistent_cache: (nil) 2024-09-11T08:06:13.683 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_size: 4096 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_size_deviation: 10 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_restart_interval: 16 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.a.smithi071.stdout: index_block_restart_interval: 1 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.a.smithi071.stdout: metadata_block_size: 4096 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.a.smithi071.stdout: partition_filters: 0 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.a.smithi071.stdout: use_delta_encoding: 1 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.a.smithi071.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.a.smithi071.stdout: whole_key_filtering: 1 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.a.smithi071.stdout: verify_compression: 0 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.a.smithi071.stdout: read_amp_bytes_per_bit: 0 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.a.smithi071.stdout: format_version: 4 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.a.smithi071.stdout: enable_index_compression: 1 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_align: 0 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compression: NoCompression 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.num_levels: 7 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T08:06:13.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T08:06:13.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T08:06:13.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T08:06:13.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T08:06:13.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T08:06:13.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:06:13.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T08:06:13.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T08:06:13.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T08:06:13.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T08:06:13.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T08:06:13.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T08:06:13.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T08:06:13.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T08:06:13.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T08:06:13.687 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T08:06:13.687 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T08:06:13.687 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T08:06:13.687 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T08:06:13.687 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T08:06:13.687 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T08:06:13.687 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T08:06:13.687 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T08:06:13.687 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T08:06:13.687 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T08:06:13.687 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T08:06:13.687 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T08:06:13.687 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T08:06:13.687 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T08:06:13.687 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T08:06:13.688 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T08:06:13.688 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T08:06:13.688 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T08:06:13.688 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T08:06:13.688 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T08:06:13.688 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T08:06:13.688 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T08:06:13.688 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T08:06:13.688 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T08:06:13.688 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T08:06:13.688 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T08:06:13.688 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.inplace_update_support: 0 2024-09-11T08:06:13.688 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T08:06:13.688 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T08:06:13.688 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T08:06:13.689 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T08:06:13.689 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.bloom_locality: 0 2024-09-11T08:06:13.689 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.max_successive_merges: 0 2024-09-11T08:06:13.689 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T08:06:13.689 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T08:06:13.689 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T08:06:13.689 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T08:06:13.689 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.ttl: 2592000 2024-09-11T08:06:13.689 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T08:06:13.689 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.enable_blob_files: false 2024-09-11T08:06:13.689 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.min_blob_size: 0 2024-09-11T08:06:13.689 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T08:06:13.689 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T08:06:13.689 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T08:06:13.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T08:06:13.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T08:06:13.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-11T08:06:13.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-11T08:06:13.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041973608632, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-11T08:06:13.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-11T08:06:13.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T08:06:13.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041973609454, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 79582, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 77814, "index_size": 230, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9759, "raw_average_key_size": 47, "raw_value_size": 72221, "raw_average_value_size": 348, "num_data_blocks": 10, "num_entries": 207, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726041973, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "ad2ab5b5-bd63-4565-b85e-fb8361e78405", "db_session_id": "1EX28L2JKN1ZS3VO3JT4"}} 2024-09-11T08:06:13.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-11T08:06:13.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041973609799, "job": 1, "event": "recovery_finished"} 2024-09-11T08:06:13.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T08:06:13.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5578edf78700 2024-09-11T08:06:13.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: DB pointer 0x5578edf48000 2024-09-11T08:06:13.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T08:06:13.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T08:06:13.691 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** DB Stats ** 2024-09-11T08:06:13.691 INFO:journalctl@ceph.mon.a.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:06:13.691 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T08:06:13.691 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T08:06:13.691 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:06:13.691 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T08:06:13.691 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T08:06:13.691 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:06:13.691 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-11T08:06:13.691 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** Compaction Stats [default] ** 2024-09-11T08:06:13.691 INFO:journalctl@ceph.mon.a.smithi071.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:06:13.691 INFO:journalctl@ceph.mon.a.smithi071.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:06:13.691 INFO:journalctl@ceph.mon.a.smithi071.stdout: L0 2/0 79.50 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 128.2 0.00 0.00 1 0.001 0 0 2024-09-11T08:06:13.691 INFO:journalctl@ceph.mon.a.smithi071.stdout: Sum 2/0 79.50 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 128.2 0.00 0.00 1 0.001 0 0 2024-09-11T08:06:13.691 INFO:journalctl@ceph.mon.a.smithi071.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 128.2 0.00 0.00 1 0.001 0 0 2024-09-11T08:06:13.692 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-11T08:06:13.692 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** Compaction Stats [default] ** 2024-09-11T08:06:13.692 INFO:journalctl@ceph.mon.a.smithi071.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:06:13.692 INFO:journalctl@ceph.mon.a.smithi071.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:06:13.692 INFO:journalctl@ceph.mon.a.smithi071.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 128.2 0.00 0.00 1 0.001 0 0 2024-09-11T08:06:13.692 INFO:journalctl@ceph.mon.a.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:06:13.692 INFO:journalctl@ceph.mon.a.smithi071.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:06:13.692 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:06:13.692 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:06:13.692 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:06:13.692 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:06:13.692 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative compaction: 0.00 GB write, 24.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:06:13.692 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval compaction: 0.00 GB write, 24.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:06:13.692 INFO:journalctl@ceph.mon.a.smithi071.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:06:13.693 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-11T08:06:13.693 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:06:13.693 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-11T08:06:13.693 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** Compaction Stats [default] ** 2024-09-11T08:06:13.693 INFO:journalctl@ceph.mon.a.smithi071.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:06:13.693 INFO:journalctl@ceph.mon.a.smithi071.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:06:13.693 INFO:journalctl@ceph.mon.a.smithi071.stdout: L0 2/0 79.50 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 128.2 0.00 0.00 1 0.001 0 0 2024-09-11T08:06:13.693 INFO:journalctl@ceph.mon.a.smithi071.stdout: Sum 2/0 79.50 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 128.2 0.00 0.00 1 0.001 0 0 2024-09-11T08:06:13.693 INFO:journalctl@ceph.mon.a.smithi071.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T08:06:13.693 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-11T08:06:13.693 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** Compaction Stats [default] ** 2024-09-11T08:06:13.693 INFO:journalctl@ceph.mon.a.smithi071.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:06:13.693 INFO:journalctl@ceph.mon.a.smithi071.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:06:13.693 INFO:journalctl@ceph.mon.a.smithi071.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 128.2 0.00 0.00 1 0.001 0 0 2024-09-11T08:06:13.694 INFO:journalctl@ceph.mon.a.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:06:13.694 INFO:journalctl@ceph.mon.a.smithi071.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:06:13.694 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:06:13.694 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:06:13.694 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:06:13.694 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:06:13.694 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative compaction: 0.00 GB write, 24.05 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:06:13.694 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:06:13.694 INFO:journalctl@ceph.mon.a.smithi071.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:06:13.694 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-11T08:06:13.694 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:06:13.694 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: starting mon.a rank 0 at public addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] at bind addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:06:13.694 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: mon.a@-1(???) e1 preinit fsid 566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:06:13.694 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: mon.a@-1(???).mds e1 new map 2024-09-11T08:06:13.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: mon.a@-1(???).mds e1 print_map 2024-09-11T08:06:13.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: e1 2024-09-11T08:06:13.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T08:06:13.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T08:06:13.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: legacy client fscid: -1 2024-09-11T08:06:13.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-11T08:06:13.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: No filesystems configured 2024-09-11T08:06:13.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T08:06:13.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:06:13.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:06:13.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:06:13.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T08:06:13.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T08:06:13.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: monmap e1: 1 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:06:13.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: fsmap 2024-09-11T08:06:13.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T08:06:13.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 ceph-mon[25834]: mgrmap e1: no daemons active 2024-09-11T08:06:13.696 INFO:teuthology.orchestra.run.smithi071.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T08:06:13.930 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 bash[25796]: e6731b5b742dd722fd8592e46902bb9992bc9df713c2cc77ea95078a242e7a9f 2024-09-11T08:06:13.930 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:13 smithi071 systemd[1]: Started Ceph mon.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:06:14.698 INFO:teuthology.orchestra.run.smithi071.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T08:06:14.699 INFO:teuthology.orchestra.run.smithi071.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:06:14.699 INFO:teuthology.orchestra.run.smithi071.stdout:Creating mgr... 2024-09-11T08:06:14.699 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T08:06:14.699 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T08:06:14.883 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-566367e6-7014-11ef-bcea-c7b262605968@mgr.y 2024-09-11T08:06:14.883 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to reset failed state of unit ceph-566367e6-7014-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-566367e6-7014-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-11T08:06:15.037 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-566367e6-7014-11ef-bcea-c7b262605968.target.wants/ceph-566367e6-7014-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-566367e6-7014-11ef-bcea-c7b262605968@.service. 2024-09-11T08:06:15.182 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:15 smithi071 systemd[1]: Starting Ceph mgr.y for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:06:15.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:15 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3560984483' entity='client.admin' 2024-09-11T08:06:15.445 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:15 smithi071 podman[26074]: 2024-09-11 08:06:15.166448933 +0000 UTC m=+0.015603393 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:06:15.445 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:15 smithi071 podman[26074]: 2024-09-11 08:06:15.287838176 +0000 UTC m=+0.136992636 container create da69cb3a1c3552a832d87ece080928af865908965ba867eec3c7348c89c09aa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T08:06:15.579 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:06:15.579 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-09-11T08:06:15.587 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:06:15.587 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-11T08:06:15.587 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-09-11T08:06:15.587 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T08:06:15.596 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:06:15.596 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-09-11T08:06:15.604 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:06:15.604 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-11T08:06:15.604 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-09-11T08:06:15.604 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T08:06:15.604 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr to start... 2024-09-11T08:06:15.604 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr... 2024-09-11T08:06:15.728 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:15 smithi071 podman[26074]: 2024-09-11 08:06:15.473376124 +0000 UTC m=+0.322530583 container init da69cb3a1c3552a832d87ece080928af865908965ba867eec3c7348c89c09aa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T08:06:15.728 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:15 smithi071 podman[26074]: 2024-09-11 08:06:15.478033702 +0000 UTC m=+0.327188163 container start da69cb3a1c3552a832d87ece080928af865908965ba867eec3c7348c89c09aa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T08:06:15.728 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:15 smithi071 bash[26074]: da69cb3a1c3552a832d87ece080928af865908965ba867eec3c7348c89c09aa2 2024-09-11T08:06:15.728 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:15 smithi071 systemd[1]: Started Ceph mgr.y for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:06:15.728 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:15 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:15.592+0000 7f37b5204ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:06:15.728 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:15 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:15.651+0000 7f37b5204ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:06:15.729 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:15 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:15.715+0000 7f37b5204ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:06:15.988 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:15 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:15.845+0000 7f37b5204ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:06:15.988 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:15 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:15.986+0000 7f37b5204ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:06:16.258 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:16 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/582625759' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T08:06:16.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:16 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:16.257+0000 7f37b5204ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:06:16.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:16 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:16.318+0000 7f37b5204ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:06:16.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:16 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:16.482+0000 7f37b5204ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:06:16.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T08:06:16.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:06:16.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsid": "566367e6-7014-11ef-bcea-c7b262605968", 2024-09-11T08:06:16.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T08:06:16.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T08:06:16.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T08:06:16.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T08:06:16.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:16.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T08:06:16.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T08:06:16.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 0 2024-09-11T08:06:16.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:06:16.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T08:06:16.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T08:06:16.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:06:16.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-11T08:06:16.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T08:06:16.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:06:16.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T08:06:16.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T08:06:16.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:16.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T08:06:16.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:06:16.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T08:06:16.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T08:06:16.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T08:06:16.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T08:06:16.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T08:06:16.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T08:06:16.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:16.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T08:06:16.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T08:06:16.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T08:06:16.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T08:06:16.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T08:06:16.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T08:06:16.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T08:06:16.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T08:06:16.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T08:06:16.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:16.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T08:06:16.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:06:16.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T08:06:16.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T08:06:16.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:16.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T08:06:16.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T08:06:16.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T08:06:16.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T08:06:16.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T08:06:16.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T08:06:16.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T08:06:16.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:06:16.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:06:16.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:16.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T08:06:16.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:06:16.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T08:06:08.949147+0000", 2024-09-11T08:06:16.809 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:06:16.809 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:16.809 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T08:06:16.809 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:06:16.809 INFO:teuthology.orchestra.run.smithi071.stdout:mgr not available, waiting (1/15)... 2024-09-11T08:06:17.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:17 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:17.328+0000 7f37b5204ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:06:17.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:17 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:17.393+0000 7f37b5204ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:06:18.383 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:18 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:18.095+0000 7f37b5204ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:06:18.383 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:18 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:18.204+0000 7f37b5204ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:06:18.384 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:18 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:18.323+0000 7f37b5204ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:06:18.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:18 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:18.382+0000 7f37b5204ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:06:18.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:18 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:18.506+0000 7f37b5204ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:06:18.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:18 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:18.571+0000 7f37b5204ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:06:18.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:18 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:18.629+0000 7f37b5204ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:06:19.038 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:18 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:18.763+0000 7f37b5204ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:06:19.038 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:18 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:18.828+0000 7f37b5204ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:06:19.038 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:18 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:18.894+0000 7f37b5204ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:06:19.038 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:18 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:18.977+0000 7f37b5204ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:06:19.038 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:19 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:19.036+0000 7f37b5204ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:06:19.324 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:19 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:19.100+0000 7f37b5204ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:06:19.635 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:19 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/4111427815' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T08:06:19.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:19 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:19.866+0000 7f37b5204ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:06:19.965 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T08:06:19.965 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:06:19.965 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsid": "566367e6-7014-11ef-bcea-c7b262605968", 2024-09-11T08:06:19.965 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T08:06:19.965 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T08:06:19.965 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T08:06:19.965 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T08:06:19.965 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:19.965 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T08:06:19.966 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T08:06:19.966 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 0 2024-09-11T08:06:19.966 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:06:19.966 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T08:06:19.966 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T08:06:19.966 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:06:19.966 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-11T08:06:19.966 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T08:06:19.966 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:06:19.966 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T08:06:19.966 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T08:06:19.966 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:19.966 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T08:06:19.966 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:06:19.967 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T08:06:19.967 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T08:06:19.967 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T08:06:19.970 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T08:06:19.971 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T08:06:19.971 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T08:06:19.971 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:19.971 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T08:06:19.971 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T08:06:19.971 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T08:06:19.971 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T08:06:19.971 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T08:06:19.971 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T08:06:19.971 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T08:06:19.971 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T08:06:19.972 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T08:06:19.972 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:19.972 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T08:06:19.972 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:06:19.972 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T08:06:19.972 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T08:06:19.972 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:19.972 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T08:06:19.972 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T08:06:19.972 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T08:06:19.972 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T08:06:19.972 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T08:06:19.972 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T08:06:19.972 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T08:06:19.973 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:06:19.973 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:06:19.973 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:19.973 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T08:06:19.973 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:06:19.973 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T08:06:08.949147+0000", 2024-09-11T08:06:19.973 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:06:19.973 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:19.973 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T08:06:19.973 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:06:19.973 INFO:teuthology.orchestra.run.smithi071.stdout:mgr not available, waiting (2/15)... 2024-09-11T08:06:20.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:20 smithi071 ceph-mon[25834]: Activating manager daemon y 2024-09-11T08:06:20.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:20 smithi071 ceph-mon[25834]: mgrmap e2: y(active, starting, since 0.00281514s) 2024-09-11T08:06:20.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:20 smithi071 ceph-mon[25834]: from='mgr.14100 172.21.15.71:0/4076243252' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:06:20.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:20 smithi071 ceph-mon[25834]: from='mgr.14100 172.21.15.71:0/4076243252' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:06:20.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:20 smithi071 ceph-mon[25834]: from='mgr.14100 172.21.15.71:0/4076243252' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:06:20.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:20 smithi071 ceph-mon[25834]: from='mgr.14100 172.21.15.71:0/4076243252' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:06:20.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:20 smithi071 ceph-mon[25834]: from='mgr.14100 172.21.15.71:0/4076243252' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:06:20.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:20 smithi071 ceph-mon[25834]: Manager daemon y is now available 2024-09-11T08:06:20.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:20 smithi071 ceph-mon[25834]: from='mgr.14100 172.21.15.71:0/4076243252' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:06:20.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:20 smithi071 ceph-mon[25834]: from='mgr.14100 172.21.15.71:0/4076243252' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:06:20.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:20 smithi071 ceph-mon[25834]: from='mgr.14100 172.21.15.71:0/4076243252' entity='mgr.y' 2024-09-11T08:06:20.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:20 smithi071 ceph-mon[25834]: from='mgr.14100 172.21.15.71:0/4076243252' entity='mgr.y' 2024-09-11T08:06:20.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:20 smithi071 ceph-mon[25834]: from='mgr.14100 172.21.15.71:0/4076243252' entity='mgr.y' 2024-09-11T08:06:22.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:21 smithi071 ceph-mon[25834]: mgrmap e3: y(active, since 1.00603s) 2024-09-11T08:06:22.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:22 smithi071 ceph-mon[25834]: mgrmap e4: y(active, since 2s) 2024-09-11T08:06:22.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:22 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/29872423' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T08:06:23.186 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T08:06:23.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:06:23.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsid": "566367e6-7014-11ef-bcea-c7b262605968", 2024-09-11T08:06:23.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T08:06:23.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T08:06:23.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T08:06:23.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T08:06:23.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:23.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T08:06:23.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T08:06:23.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 0 2024-09-11T08:06:23.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:06:23.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T08:06:23.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T08:06:23.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:06:23.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-11T08:06:23.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T08:06:23.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:06:23.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T08:06:23.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T08:06:23.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:23.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T08:06:23.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:06:23.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T08:06:23.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T08:06:23.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T08:06:23.194 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T08:06:23.194 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T08:06:23.194 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T08:06:23.194 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:23.194 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T08:06:23.194 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T08:06:23.194 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T08:06:23.194 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T08:06:23.194 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T08:06:23.194 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T08:06:23.194 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T08:06:23.194 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T08:06:23.194 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T08:06:23.195 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:23.195 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T08:06:23.195 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:06:23.195 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T08:06:23.195 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T08:06:23.195 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:23.195 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T08:06:23.195 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T08:06:23.195 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T08:06:23.195 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T08:06:23.195 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T08:06:23.195 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T08:06:23.195 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T08:06:23.195 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:06:23.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:06:23.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:23.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T08:06:23.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:06:23.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T08:06:08.949147+0000", 2024-09-11T08:06:23.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:06:23.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:06:23.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T08:06:23.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:06:23.196 INFO:teuthology.orchestra.run.smithi071.stdout:mgr is available 2024-09-11T08:06:24.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:23 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/353765334' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T08:06:24.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:23 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/353765334' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T08:06:24.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T08:06:24.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T08:06:24.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout fsid = 566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:06:24.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T08:06:24.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T08:06:24.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T08:06:24.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T08:06:24.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T08:06:24.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T08:06:24.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T08:06:24.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T08:06:24.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T08:06:24.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T08:06:24.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T08:06:24.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T08:06:24.334 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling cephadm module... 2024-09-11T08:06:25.318 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:25 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3465573758' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T08:06:25.319 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:25 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: ignoring --setuser ceph since I am not root 2024-09-11T08:06:25.319 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:25 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: ignoring --setgroup ceph since I am not root 2024-09-11T08:06:25.319 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:25 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:25.150+0000 7fbc4b4d2ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:06:25.319 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:25 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:25.204+0000 7fbc4b4d2ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:06:25.319 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:25 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:25.270+0000 7fbc4b4d2ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:06:25.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:25 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:25.394+0000 7fbc4b4d2ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:06:25.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:25 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:25.531+0000 7fbc4b4d2ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:06:26.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:26 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3465573758' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T08:06:26.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:26 smithi071 ceph-mon[25834]: mgrmap e5: y(active, since 5s) 2024-09-11T08:06:26.195 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:25 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:25.788+0000 7fbc4b4d2ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:06:26.195 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:25 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:25.847+0000 7fbc4b4d2ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:06:26.195 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:26 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:26.005+0000 7fbc4b4d2ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:06:27.195 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:26 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:26.851+0000 7fbc4b4d2ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:06:27.195 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:26 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:26.914+0000 7fbc4b4d2ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:06:27.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:27 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:27.640+0000 7fbc4b4d2ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:06:27.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:27 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:27.758+0000 7fbc4b4d2ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:06:27.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:27 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:27.885+0000 7fbc4b4d2ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:06:28.205 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:27 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:27.946+0000 7fbc4b4d2ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:06:28.205 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:28 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:28.076+0000 7fbc4b4d2ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:06:28.205 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:28 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:28.144+0000 7fbc4b4d2ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:06:28.205 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:28 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:28.203+0000 7fbc4b4d2ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:06:28.474 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:28 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:28.341+0000 7fbc4b4d2ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:06:28.474 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:28 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:28.407+0000 7fbc4b4d2ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:06:28.475 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:28 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:28.473+0000 7fbc4b4d2ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:06:28.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:28 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:28.553+0000 7fbc4b4d2ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:06:28.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:28 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:28.610+0000 7fbc4b4d2ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:06:28.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:28 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:28.671+0000 7fbc4b4d2ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:06:29.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:29.433+0000 7fbc4b4d2ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:06:29.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-mon[25834]: Active manager daemon y restarted 2024-09-11T08:06:29.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-mon[25834]: Activating manager daemon y 2024-09-11T08:06:29.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-mon[25834]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T08:06:29.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-mon[25834]: mgrmap e6: y(active, starting, since 0.00324534s) 2024-09-11T08:06:29.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:06:29.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:06:29.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:06:29.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:06:29.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:06:29.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-mon[25834]: Manager daemon y is now available 2024-09-11T08:06:29.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' 2024-09-11T08:06:29.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' 2024-09-11T08:06:29.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:29.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:29.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:06:29.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:29 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:06:30.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:30 smithi071 ceph-mon[25834]: Found migration_current of "None". Setting to last migration. 2024-09-11T08:06:30.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:30 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3516265992' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T08:06:30.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:30 smithi071 ceph-mon[25834]: mgrmap e7: y(active, since 1.00484s) 2024-09-11T08:06:31.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:31.392+0000 7fbbfd8c0640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:06:31] ENGINE Error in HTTPServer.serve 2024-09-11T08:06:31.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: Traceback (most recent call last): 2024-09-11T08:06:31.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:06:31.696 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: self._connections.run(self.expiration_interval) 2024-09-11T08:06:31.696 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:06:31.696 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: self._run(expiration_interval) 2024-09-11T08:06:31.696 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:06:31.696 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:06:31.696 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:06:31.696 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:06:31.696 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:06:31.696 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: s = self.context.wrap_socket( 2024-09-11T08:06:31.696 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:06:31.696 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: return self.sslsocket_class._create( 2024-09-11T08:06:31.696 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:06:31.696 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: self.do_handshake() 2024-09-11T08:06:31.696 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:06:31.696 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: self._sslobj.do_handshake() 2024-09-11T08:06:31.697 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:06:31.697 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:32.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:32 smithi071 ceph-mon[25834]: [11/Sep/2024:08:06:31] ENGINE Bus STARTING 2024-09-11T08:06:32.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:32 smithi071 ceph-mon[25834]: [11/Sep/2024:08:06:31] ENGINE Serving on https://172.21.15.71:7150 2024-09-11T08:06:32.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:32 smithi071 ceph-mon[25834]: [11/Sep/2024:08:06:31] ENGINE Bus STARTED 2024-09-11T08:06:32.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:32 smithi071 ceph-mon[25834]: [11/Sep/2024:08:06:31] ENGINE Error in HTTPServer.serve 2024-09-11T08:06:32.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: Traceback (most recent call last): 2024-09-11T08:06:32.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:06:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:06:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:06:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._run(expiration_interval) 2024-09-11T08:06:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:06:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:06:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:06:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:06:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:06:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: s = self.context.wrap_socket( 2024-09-11T08:06:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:06:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: return self.sslsocket_class._create( 2024-09-11T08:06:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:06:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: self.do_handshake() 2024-09-11T08:06:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:06:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._sslobj.do_handshake() 2024-09-11T08:06:32.697 INFO:journalctl@ceph.mon.a.smithi071.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:06:32.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:32 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' 2024-09-11T08:06:32.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:32 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:33.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:33 smithi071 ceph-mon[25834]: mgrmap e8: y(active, since 2s) 2024-09-11T08:06:36.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:06:36.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 6, 2024-09-11T08:06:36.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T08:06:36.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T08:06:36.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T08:06:36.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:06:36.368 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for the mgr to restart... 2024-09-11T08:06:36.368 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr epoch 6... 2024-09-11T08:06:37.996 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:06:37.996 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 8, 2024-09-11T08:06:37.996 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T08:06:37.997 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:06:37.997 INFO:teuthology.orchestra.run.smithi071.stdout:mgr epoch 6 is available 2024-09-11T08:06:37.997 INFO:teuthology.orchestra.run.smithi071.stdout:Setting orchestrator backend to cephadm... 2024-09-11T08:06:38.337 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:38 smithi071 ceph-mon[25834]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T08:06:38.337 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:38 smithi071 ceph-mon[25834]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T08:06:39.780 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:39 smithi071 ceph-mon[25834]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:39.780 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:39 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' 2024-09-11T08:06:39.780 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:39 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:40.326 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T08:06:40.326 INFO:teuthology.orchestra.run.smithi071.stdout:Generating ssh key... 2024-09-11T08:06:40.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:40 smithi071 ceph-mon[25834]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:41.421 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: Generating public/private rsa key pair. 2024-09-11T08:06:41.422 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: Your identification has been saved in /tmp/tmpf99sssvb/key 2024-09-11T08:06:41.422 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: Your public key has been saved in /tmp/tmpf99sssvb/key.pub 2024-09-11T08:06:41.422 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: The key fingerprint is: 2024-09-11T08:06:41.422 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: SHA256:6Qhk9wkulGUdchKYVoFH5Km6DSb7fYEU9nCVjnY+tl4 ceph-566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:06:41.422 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: The key's randomart image is: 2024-09-11T08:06:41.422 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: +---[RSA 3072]----+ 2024-09-11T08:06:41.422 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: | B@=+o | 2024-09-11T08:06:41.422 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: | O=o=o | 2024-09-11T08:06:41.422 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: | o=*=o | 2024-09-11T08:06:41.422 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: | +.++ooo | 2024-09-11T08:06:41.422 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: | .+o.oS | 2024-09-11T08:06:41.423 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: | ..o.o+ | 2024-09-11T08:06:41.423 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: |. + .o.oE | 2024-09-11T08:06:41.423 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: | + = . .. | 2024-09-11T08:06:41.423 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: |..o o. .. | 2024-09-11T08:06:41.423 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: +----[SHA256]-----+ 2024-09-11T08:06:41.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-mon[25834]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:41.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-mon[25834]: Generating ssh key... 2024-09-11T08:06:41.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' 2024-09-11T08:06:41.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:41 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' 2024-09-11T08:06:42.896 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCd0YReIxSVRg9FbN0S0cYaO/Z3m2g09W5XTCNpSa/HCf8zm+kCvM1R+682VYO5jDiznrS1/pszPTw0+u8qMDR8bZf7pF1a3idsjKswEPiP/fJjiDrTlQNCOO9wmWGK11xNwW7e30EkbFgrbYkXmeQSrsnQ6nvOoZ4PRUxUlCGKNjPIiXh2A96wUp//GI9+PUvWRVLyxdu46wtwTDLJgWSh/Gm8/EfFHOoH43zmcn4eWfnX6ruRHk5r3mOMElp+FuYfvdOljMYgTj6AA4Ndk3qzeFSkqg6NYyjsR2mtPdxtJCnAtb+K6sn7kTbCbCPq0jPbIw1Yloj19oKHtKGh8MT19rn9TVl6jlaGkpHqO/7GCKRLs/1uTGZFfQi9Kid954CxNYmtlDK+2Q4fGEfyUkGXabk1xTBtL5R8y1Pt5YaHy/nRGNfQVFRr9AoHwbklm5QKiEypY1JbywnHBR3V8WfDLbTJ5qXMxRipJ2+0SErw8D+jwKlsaDtYSOrVoCGsEXk= ceph-566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:06:42.897 INFO:teuthology.orchestra.run.smithi071.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T08:06:42.897 INFO:teuthology.orchestra.run.smithi071.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T08:06:42.906 INFO:teuthology.orchestra.run.smithi071.stdout:Adding host smithi071... 2024-09-11T08:06:43.147 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:43 smithi071 ceph-mon[25834]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:44.335 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:44 smithi071 ceph-mon[25834]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi071", "addr": "172.21.15.71", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:45.280 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:45 smithi071 ceph-mon[25834]: Deploying cephadm binary to smithi071 2024-09-11T08:06:46.845 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Added host 'smithi071' with addr '172.21.15.71' 2024-09-11T08:06:46.845 INFO:teuthology.orchestra.run.smithi071.stdout:Deploying unmanaged mon service... 2024-09-11T08:06:47.463 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:47 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' 2024-09-11T08:06:47.463 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:47 smithi071 ceph-mon[25834]: Added host smithi071 2024-09-11T08:06:47.464 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:47 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:48.303 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T08:06:48.303 INFO:teuthology.orchestra.run.smithi071.stdout:Deploying unmanaged mgr service... 2024-09-11T08:06:48.612 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:48 smithi071 ceph-mon[25834]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:48.613 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:48 smithi071 ceph-mon[25834]: Saving service mon spec with placement count:5 2024-09-11T08:06:48.613 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:48 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' 2024-09-11T08:06:50.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T08:06:50.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:50 smithi071 ceph-mon[25834]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:06:50.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:50 smithi071 ceph-mon[25834]: Saving service mgr spec with placement count:2 2024-09-11T08:06:50.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:50 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' 2024-09-11T08:06:50.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:50 smithi071 ceph-mon[25834]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:06:50.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:50 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' 2024-09-11T08:06:51.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:51 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' 2024-09-11T08:06:51.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:51 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3556681138' entity='client.admin' 2024-09-11T08:06:53.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:52 smithi071 ceph-mon[25834]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:06:53.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:52 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' 2024-09-11T08:06:53.572 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling the dashboard module... 2024-09-11T08:06:54.081 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:53 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/93371265' entity='client.admin' 2024-09-11T08:06:54.081 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:53 smithi071 ceph-mon[25834]: from='mgr.14118 172.21.15.71:0/3980314587' entity='mgr.y' 2024-09-11T08:06:55.154 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:55 smithi071 ceph-mon[25834]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:06:55.154 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:55 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3728555412' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T08:06:55.154 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:55 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: ignoring --setuser ceph since I am not root 2024-09-11T08:06:55.154 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:55 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: ignoring --setgroup ceph since I am not root 2024-09-11T08:06:55.154 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:55 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:55.152+0000 7fa43cd91ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:06:55.445 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:55 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:55.208+0000 7fa43cd91ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:06:55.445 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:55 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:55.272+0000 7fa43cd91ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:06:55.445 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:55 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:55.394+0000 7fa43cd91ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:06:55.790 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:55 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:55.530+0000 7fa43cd91ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:06:55.790 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:55 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:55.789+0000 7fa43cd91ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:06:56.157 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:56 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3728555412' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T08:06:56.157 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:56 smithi071 ceph-mon[25834]: mgrmap e9: y(active, since 25s) 2024-09-11T08:06:56.157 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:55 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:55.848+0000 7fa43cd91ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:06:56.157 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:56 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:56.011+0000 7fa43cd91ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:06:56.957 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:56 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:56.888+0000 7fa43cd91ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:06:56.957 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:56 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:56.956+0000 7fa43cd91ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:06:57.283 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:57 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/1681305403' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T08:06:57.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:06:57.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T08:06:57.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T08:06:57.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T08:06:57.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T08:06:57.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:06:57.287 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for the mgr to restart... 2024-09-11T08:06:57.287 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr epoch 9... 2024-09-11T08:06:57.792 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:57 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:57.683+0000 7fa43cd91ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:06:58.104 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:57 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:57.800+0000 7fa43cd91ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:06:58.104 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:57 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:57.919+0000 7fa43cd91ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:06:58.104 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:57 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:57.977+0000 7fa43cd91ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:06:58.104 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:58 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:58.102+0000 7fa43cd91ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:06:58.363 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:58 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:58.169+0000 7fa43cd91ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:06:58.363 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:58 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:58.227+0000 7fa43cd91ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:06:58.363 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:58 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:58.361+0000 7fa43cd91ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:06:58.634 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:58 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:58.425+0000 7fa43cd91ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:06:58.634 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:58 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:58.490+0000 7fa43cd91ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:06:58.634 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:58 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:58.572+0000 7fa43cd91ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:06:58.634 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:58 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:58.632+0000 7fa43cd91ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:06:58.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:58 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:58.692+0000 7fa43cd91ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:06:59.762 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:06:59 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:06:59.467+0000 7fa43cd91ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:06:59.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:59 smithi071 ceph-mon[25834]: Active manager daemon y restarted 2024-09-11T08:06:59.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:59 smithi071 ceph-mon[25834]: Activating manager daemon y 2024-09-11T08:06:59.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:59 smithi071 ceph-mon[25834]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T08:06:59.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:59 smithi071 ceph-mon[25834]: mgrmap e10: y(active, starting, since 0.00333062s) 2024-09-11T08:06:59.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:59 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:06:59.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:59 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:06:59.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:59 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:06:59.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:59 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:06:59.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:59 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:06:59.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:59 smithi071 ceph-mon[25834]: Manager daemon y is now available 2024-09-11T08:06:59.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:59 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:59.764 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:59 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:06:59.764 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:06:59 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:07:00.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:07:00.608+0000 7fa3ed8ee640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:07:00] ENGINE Error in HTTPServer.serve 2024-09-11T08:07:00.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: Traceback (most recent call last): 2024-09-11T08:07:00.946 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:07:00.946 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: self._connections.run(self.expiration_interval) 2024-09-11T08:07:00.946 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:07:00.946 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: self._run(expiration_interval) 2024-09-11T08:07:00.946 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:07:00.946 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:07:00.946 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:07:00.946 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:07:00.946 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:07:00.946 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: s = self.context.wrap_socket( 2024-09-11T08:07:00.946 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:07:00.946 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: return self.sslsocket_class._create( 2024-09-11T08:07:00.946 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:07:00.946 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: self.do_handshake() 2024-09-11T08:07:00.947 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:07:00.947 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: self._sslobj.do_handshake() 2024-09-11T08:07:00.947 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:07:00.947 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:07:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:07:01.443 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:07:01.443 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T08:07:01.443 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T08:07:01.443 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:07:01.443 INFO:teuthology.orchestra.run.smithi071.stdout:mgr epoch 9 is available 2024-09-11T08:07:01.443 INFO:teuthology.orchestra.run.smithi071.stdout:Generating a dashboard self-signed certificate... 2024-09-11T08:07:01.694 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:01 smithi071 ceph-mon[25834]: mgrmap e11: y(active, since 1.00574s) 2024-09-11T08:07:01.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:01 smithi071 ceph-mon[25834]: [11/Sep/2024:08:07:00] ENGINE Bus STARTING 2024-09-11T08:07:01.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:01 smithi071 ceph-mon[25834]: [11/Sep/2024:08:07:00] ENGINE Serving on https://172.21.15.71:7150 2024-09-11T08:07:01.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:01 smithi071 ceph-mon[25834]: [11/Sep/2024:08:07:00] ENGINE Bus STARTED 2024-09-11T08:07:01.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:01 smithi071 ceph-mon[25834]: [11/Sep/2024:08:07:00] ENGINE Error in HTTPServer.serve 2024-09-11T08:07:01.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: Traceback (most recent call last): 2024-09-11T08:07:01.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:07:01.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:07:01.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:07:01.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._run(expiration_interval) 2024-09-11T08:07:01.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:07:01.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:07:01.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:07:01.695 INFO:journalctl@ceph.mon.a.smithi071.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:07:01.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:07:01.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: s = self.context.wrap_socket( 2024-09-11T08:07:01.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:07:01.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: return self.sslsocket_class._create( 2024-09-11T08:07:01.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:07:01.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: self.do_handshake() 2024-09-11T08:07:01.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:07:01.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._sslobj.do_handshake() 2024-09-11T08:07:01.696 INFO:journalctl@ceph.mon.a.smithi071.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:07:01.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:01 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:01.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:01 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:02.815 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T08:07:02.815 INFO:teuthology.orchestra.run.smithi071.stdout:Creating initial admin user... 2024-09-11T08:07:03.149 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:02 smithi071 ceph-mon[25834]: mgrmap e12: y(active, since 2s) 2024-09-11T08:07:03.149 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:02 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:03.149 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:02 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:03.149 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:02 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:04.156 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:03 smithi071 ceph-mon[25834]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:07:04.451 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$oZjQFLb4dBsHSVd9eqH3Q.cH4/ZdX6ePpZRbN2156xVaJYKqn3IrS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726042023, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T08:07:04.451 INFO:teuthology.orchestra.run.smithi071.stdout:Fetching dashboard port number... 2024-09-11T08:07:05.113 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:04 smithi071 ceph-mon[25834]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:07:05.113 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:04 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:06.065 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:05 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2236608795' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T08:07:06.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T08:07:06.077 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:07:06.077 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-09-11T08:07:06.084 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:07:06.084 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-11T08:07:06.084 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-09-11T08:07:06.084 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T08:07:06.085 INFO:teuthology.orchestra.run.smithi071.stdout:Ceph Dashboard is now available at: 2024-09-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi071.stdout: URL: https://smithi071.front.sepia.ceph.com:8443/ 2024-09-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi071.stdout: User: admin 2024-09-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi071.stdout: Password: hfn0ki6m54 2024-09-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi071.stdout:Saving cluster configuration to /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config directory 2024-09-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling autotune for osd_memory_target 2024-09-11T08:07:06.988 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:06 smithi071 ceph-mon[25834]: mgrmap e13: y(active, since 6s) 2024-09-11T08:07:08.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T08:07:08.606 INFO:teuthology.orchestra.run.smithi071.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T08:07:08.606 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:07:08.606 INFO:teuthology.orchestra.run.smithi071.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:07:08.606 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:07:08.606 INFO:teuthology.orchestra.run.smithi071.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T08:07:08.606 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:07:08.606 INFO:teuthology.orchestra.run.smithi071.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-11T08:07:08.606 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:07:08.606 INFO:teuthology.orchestra.run.smithi071.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T08:07:08.606 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:07:08.606 INFO:teuthology.orchestra.run.smithi071.stdout: ceph telemetry on 2024-09-11T08:07:08.606 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:07:08.607 INFO:teuthology.orchestra.run.smithi071.stdout:For more information see: 2024-09-11T08:07:08.607 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:07:08.607 INFO:teuthology.orchestra.run.smithi071.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-11T08:07:08.607 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:07:08.607 INFO:teuthology.orchestra.run.smithi071.stdout:Bootstrap complete. 2024-09-11T08:07:08.642 INFO:tasks.cephadm:Fetching config... 2024-09-11T08:07:08.643 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T08:07:08.643 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T08:07:08.659 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T08:07:08.659 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T08:07:08.659 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T08:07:08.714 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T08:07:08.714 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T08:07:08.715 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T08:07:08.780 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T08:07:08.780 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T08:07:08.780 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T08:07:08.836 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T08:07:08.836 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCd0YReIxSVRg9FbN0S0cYaO/Z3m2g09W5XTCNpSa/HCf8zm+kCvM1R+682VYO5jDiznrS1/pszPTw0+u8qMDR8bZf7pF1a3idsjKswEPiP/fJjiDrTlQNCOO9wmWGK11xNwW7e30EkbFgrbYkXmeQSrsnQ6nvOoZ4PRUxUlCGKNjPIiXh2A96wUp//GI9+PUvWRVLyxdu46wtwTDLJgWSh/Gm8/EfFHOoH43zmcn4eWfnX6ruRHk5r3mOMElp+FuYfvdOljMYgTj6AA4Ndk3qzeFSkqg6NYyjsR2mtPdxtJCnAtb+K6sn7kTbCbCPq0jPbIw1Yloj19oKHtKGh8MT19rn9TVl6jlaGkpHqO/7GCKRLs/1uTGZFfQi9Kid954CxNYmtlDK+2Q4fGEfyUkGXabk1xTBtL5R8y1Pt5YaHy/nRGNfQVFRr9AoHwbklm5QKiEypY1JbywnHBR3V8WfDLbTJ5qXMxRipJ2+0SErw8D+jwKlsaDtYSOrVoCGsEXk= ceph-566367e6-7014-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T08:07:08.926 INFO:teuthology.orchestra.run.smithi071.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCd0YReIxSVRg9FbN0S0cYaO/Z3m2g09W5XTCNpSa/HCf8zm+kCvM1R+682VYO5jDiznrS1/pszPTw0+u8qMDR8bZf7pF1a3idsjKswEPiP/fJjiDrTlQNCOO9wmWGK11xNwW7e30EkbFgrbYkXmeQSrsnQ6nvOoZ4PRUxUlCGKNjPIiXh2A96wUp//GI9+PUvWRVLyxdu46wtwTDLJgWSh/Gm8/EfFHOoH43zmcn4eWfnX6ruRHk5r3mOMElp+FuYfvdOljMYgTj6AA4Ndk3qzeFSkqg6NYyjsR2mtPdxtJCnAtb+K6sn7kTbCbCPq0jPbIw1Yloj19oKHtKGh8MT19rn9TVl6jlaGkpHqO/7GCKRLs/1uTGZFfQi9Kid954CxNYmtlDK+2Q4fGEfyUkGXabk1xTBtL5R8y1Pt5YaHy/nRGNfQVFRr9AoHwbklm5QKiEypY1JbywnHBR3V8WfDLbTJ5qXMxRipJ2+0SErw8D+jwKlsaDtYSOrVoCGsEXk= ceph-566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:07:08.940 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCd0YReIxSVRg9FbN0S0cYaO/Z3m2g09W5XTCNpSa/HCf8zm+kCvM1R+682VYO5jDiznrS1/pszPTw0+u8qMDR8bZf7pF1a3idsjKswEPiP/fJjiDrTlQNCOO9wmWGK11xNwW7e30EkbFgrbYkXmeQSrsnQ6nvOoZ4PRUxUlCGKNjPIiXh2A96wUp//GI9+PUvWRVLyxdu46wtwTDLJgWSh/Gm8/EfFHOoH43zmcn4eWfnX6ruRHk5r3mOMElp+FuYfvdOljMYgTj6AA4Ndk3qzeFSkqg6NYyjsR2mtPdxtJCnAtb+K6sn7kTbCbCPq0jPbIw1Yloj19oKHtKGh8MT19rn9TVl6jlaGkpHqO/7GCKRLs/1uTGZFfQi9Kid954CxNYmtlDK+2Q4fGEfyUkGXabk1xTBtL5R8y1Pt5YaHy/nRGNfQVFRr9AoHwbklm5QKiEypY1JbywnHBR3V8WfDLbTJ5qXMxRipJ2+0SErw8D+jwKlsaDtYSOrVoCGsEXk= ceph-566367e6-7014-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T08:07:09.027 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCd0YReIxSVRg9FbN0S0cYaO/Z3m2g09W5XTCNpSa/HCf8zm+kCvM1R+682VYO5jDiznrS1/pszPTw0+u8qMDR8bZf7pF1a3idsjKswEPiP/fJjiDrTlQNCOO9wmWGK11xNwW7e30EkbFgrbYkXmeQSrsnQ6nvOoZ4PRUxUlCGKNjPIiXh2A96wUp//GI9+PUvWRVLyxdu46wtwTDLJgWSh/Gm8/EfFHOoH43zmcn4eWfnX6ruRHk5r3mOMElp+FuYfvdOljMYgTj6AA4Ndk3qzeFSkqg6NYyjsR2mtPdxtJCnAtb+K6sn7kTbCbCPq0jPbIw1Yloj19oKHtKGh8MT19rn9TVl6jlaGkpHqO/7GCKRLs/1uTGZFfQi9Kid954CxNYmtlDK+2Q4fGEfyUkGXabk1xTBtL5R8y1Pt5YaHy/nRGNfQVFRr9AoHwbklm5QKiEypY1JbywnHBR3V8WfDLbTJ5qXMxRipJ2+0SErw8D+jwKlsaDtYSOrVoCGsEXk= ceph-566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:07:09.040 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T08:07:09.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:09 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2919270554' entity='client.admin' 2024-09-11T08:07:09.287 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:11.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:11 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:11.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:11 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:11.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:11 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:11.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:11 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:11.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:11 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:11.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:11 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:11.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:11 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2631119392' entity='client.admin' 2024-09-11T08:07:11.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:11 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:11.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:11 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:12.341 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T08:07:12.341 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T08:07:12.658 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:15.832 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-09-11T08:07:15.833 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T08:07:15.833 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-09-11T08:07:15.853 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T08:07:15.853 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:07:15.912 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-09-11T08:07:15.913 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch host add smithi196 2024-09-11T08:07:16.149 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:16.402 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:16 smithi071 ceph-mon[25834]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:07:16.402 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:16 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:16.402 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:16 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:16.402 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:16 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:16.403 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:16 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:16.403 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:16 smithi071 ceph-mon[25834]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-11T08:07:16.403 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:16 smithi071 ceph-mon[25834]: Updating smithi071:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:16.403 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:16 smithi071 ceph-mon[25834]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:07:16.403 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:16 smithi071 ceph-mon[25834]: Updating smithi071:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:07:16.403 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:16 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:16.403 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:16 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:16.403 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:16 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:18.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:18 smithi071 ceph-mon[25834]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:07:19.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:19 smithi071 ceph-mon[25834]: Deploying cephadm binary to smithi196 2024-09-11T08:07:20.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:20 smithi071 ceph-mon[25834]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:20.738 INFO:teuthology.orchestra.run.smithi071.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-09-11T08:07:21.306 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T08:07:21.540 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:21.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:21.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:21 smithi071 ceph-mon[25834]: Added host smithi196 2024-09-11T08:07:21.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:21.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:21.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:22.853 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:22 smithi071 ceph-mon[25834]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:22.854 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:22 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:23.228 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:07:23.228 INFO:teuthology.orchestra.run.smithi071.stdout:[{"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-09-11T08:07:23.809 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T08:07:23.809 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T08:07:24.041 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:25.140 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:24 smithi071 ceph-mon[25834]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:07:25.140 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:24 smithi071 ceph-mon[25834]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:26.118 INFO:teuthology.orchestra.run.smithi071.stderr:adjusted tunables profile to default 2024-09-11T08:07:26.429 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:26.429 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:26.429 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:26.429 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:26.429 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:26.429 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:26.429 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:26.429 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:26 smithi071 ceph-mon[25834]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-11T08:07:26.430 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:26 smithi071 ceph-mon[25834]: Updating smithi196:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:26.430 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:26 smithi071 ceph-mon[25834]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:26.430 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:26 smithi071 ceph-mon[25834]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:07:26.430 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:26 smithi071 ceph-mon[25834]: Updating smithi196:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:07:26.430 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:26 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2998080317' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T08:07:26.430 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:26.430 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:26.430 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:26.744 INFO:tasks.cephadm:Adding mon.a on smithi071 2024-09-11T08:07:26.744 INFO:tasks.cephadm:Adding mon.c on smithi071 2024-09-11T08:07:26.745 INFO:tasks.cephadm:Adding mon.b on smithi196 2024-09-11T08:07:26.745 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi071:172.21.15.71=a;smithi071:[v2:172.21.15.71:3301,v1:172.21.15.71:6790]=c;smithi196:172.21.15.196=b' 2024-09-11T08:07:26.978 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:27.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:27 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2998080317' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T08:07:27.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:27 smithi071 ceph-mon[25834]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:07:28.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:28 smithi071 ceph-mon[25834]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:28.612 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-09-11T08:07:29.230 DEBUG:teuthology.orchestra.run.smithi071:mon.c> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T08:07:29.233 DEBUG:teuthology.orchestra.run.smithi196:mon.b> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T08:07:29.235 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T08:07:29.235 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T08:07:29.874 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:29.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:29 smithi071 ceph-mon[25834]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi071:172.21.15.71=a;smithi071:[v2:172.21.15.71:3301,v1:172.21.15.71:6790]=c;smithi196:172.21.15.196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:07:29.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:29 smithi071 ceph-mon[25834]: Saving service mon spec with placement smithi071:172.21.15.71=a;smithi071:[v2:172.21.15.71:3301,v1:172.21.15.71:6790]=c;smithi196:172.21.15.196=b;count:3 2024-09-11T08:07:29.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:29 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:29.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:29 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:29.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:29 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:29.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:29 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:29.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:29 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:29.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:29 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:07:29.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:29 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:29.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:29 smithi071 ceph-mon[25834]: Deploying daemon mon.b on smithi196 2024-09-11T08:07:30.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:30 smithi071 ceph-mon[25834]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:32.663 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T08:07:32.663 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"566367e6-7014-11ef-bcea-c7b262605968","modified":"2024-09-11T08:06:06.240802Z","created":"2024-09-11T08:06:06.240802Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T08:07:32.664 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-09-11T08:07:32.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:32 smithi071 ceph-mon[25834]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:33.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:33 smithi071 ceph-mon[25834]: from='client.? 172.21.15.196:0/2568335423' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:07:34.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 podman[30639]: 2024-09-11 08:07:34.077713466 +0000 UTC m=+0.127951111 container create 218d41aee37ff08574dbdf28bd06fd6169dc0cc98449cd5380691af70aa982ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-b, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:07:34.570 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T08:07:34.571 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T08:07:34.595 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 podman[30639]: 2024-09-11 08:07:34.2715613 +0000 UTC m=+0.321798903 container init 218d41aee37ff08574dbdf28bd06fd6169dc0cc98449cd5380691af70aa982ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-b, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T08:07:34.595 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 podman[30639]: 2024-09-11 08:07:34.276209928 +0000 UTC m=+0.326447543 container start 218d41aee37ff08574dbdf28bd06fd6169dc0cc98449cd5380691af70aa982ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS) 2024-09-11T08:07:34.595 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T08:07:34.595 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 7 2024-09-11T08:07:34.595 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: pidfile_write: ignore empty --pid-file 2024-09-11T08:07:34.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: load: jerasure load: lrc 2024-09-11T08:07:34.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: RocksDB version: 6.15.5 2024-09-11T08:07:34.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T08:07:34.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Compile date Sep 10 2024 2024-09-11T08:07:34.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: DB SUMMARY 2024-09-11T08:07:34.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: DB Session ID: W91NS9FJVQ5DCH06HEBM 2024-09-11T08:07:34.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: CURRENT file: CURRENT 2024-09-11T08:07:34.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T08:07:34.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-11T08:07:34.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-11T08:07:34.596 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-11T08:07:34.597 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.error_if_exists: 0 2024-09-11T08:07:34.597 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.create_if_missing: 0 2024-09-11T08:07:34.597 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.paranoid_checks: 1 2024-09-11T08:07:34.597 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T08:07:34.597 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.env: 0x559a172c5440 2024-09-11T08:07:34.597 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.fs: Posix File System 2024-09-11T08:07:34.598 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.info_log: 0x559a17c66ec0 2024-09-11T08:07:34.598 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T08:07:34.598 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.statistics: (nil) 2024-09-11T08:07:34.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.use_fsync: 0 2024-09-11T08:07:34.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_log_file_size: 0 2024-09-11T08:07:34.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T08:07:34.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T08:07:34.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T08:07:34.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T08:07:34.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.allow_fallocate: 1 2024-09-11T08:07:34.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T08:07:34.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T08:07:34.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.use_direct_reads: 0 2024-09-11T08:07:34.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T08:07:34.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T08:07:34.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.db_log_dir: 2024-09-11T08:07:34.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-09-11T08:07:34.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T08:07:34.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T08:07:34.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T08:07:34.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T08:07:34.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T08:07:34.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T08:07:34.604 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T08:07:34.604 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T08:07:34.604 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.write_buffer_manager: 0x559a17d247e0 2024-09-11T08:07:34.604 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T08:07:34.604 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T08:07:34.604 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T08:07:34.604 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T08:07:34.605 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T08:07:34.605 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T08:07:34.605 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T08:07:34.605 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T08:07:34.605 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T08:07:34.605 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.unordered_write: 0 2024-09-11T08:07:34.605 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T08:07:34.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T08:07:34.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T08:07:34.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T08:07:34.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.row_cache: None 2024-09-11T08:07:34.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.wal_filter: None 2024-09-11T08:07:34.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T08:07:34.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T08:07:34.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.preserve_deletes: 0 2024-09-11T08:07:34.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.two_write_queues: 0 2024-09-11T08:07:34.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T08:07:34.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.atomic_flush: 0 2024-09-11T08:07:34.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T08:07:34.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T08:07:34.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T08:07:34.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.log_readahead_size: 0 2024-09-11T08:07:34.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T08:07:34.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T08:07:34.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T08:07:34.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T08:07:34.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T08:07:34.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T08:07:34.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_background_jobs: 2 2024-09-11T08:07:34.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_background_compactions: -1 2024-09-11T08:07:34.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_subcompactions: 1 2024-09-11T08:07:34.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T08:07:34.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T08:07:34.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T08:07:34.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T08:07:34.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T08:07:34.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T08:07:34.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T08:07:34.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T08:07:34.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_open_files: -1 2024-09-11T08:07:34.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T08:07:34.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T08:07:34.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T08:07:34.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T08:07:34.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_background_flushes: -1 2024-09-11T08:07:34.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Compression algorithms supported: 2024-09-11T08:07:34.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: kZSTD supported: 0 2024-09-11T08:07:34.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: kXpressCompression supported: 0 2024-09-11T08:07:34.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: kBZip2Compression supported: 0 2024-09-11T08:07:34.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T08:07:34.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: kLZ4Compression supported: 1 2024-09-11T08:07:34.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: kZlibCompression supported: 1 2024-09-11T08:07:34.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T08:07:34.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: kSnappyCompression supported: 1 2024-09-11T08:07:34.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T08:07:34.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2024-09-11T08:07:34.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T08:07:34.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T08:07:34.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.merge_operator: 2024-09-11T08:07:34.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compaction_filter: None 2024-09-11T08:07:34.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T08:07:34.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T08:07:34.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T08:07:34.610 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T08:07:34.610 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559a17c0da38) 2024-09-11T08:07:34.610 INFO:journalctl@ceph.mon.b.smithi196.stdout: cache_index_and_filter_blocks: 1 2024-09-11T08:07:34.610 INFO:journalctl@ceph.mon.b.smithi196.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T08:07:34.610 INFO:journalctl@ceph.mon.b.smithi196.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T08:07:34.610 INFO:journalctl@ceph.mon.b.smithi196.stdout: pin_top_level_index_and_filter: 1 2024-09-11T08:07:34.610 INFO:journalctl@ceph.mon.b.smithi196.stdout: index_type: 0 2024-09-11T08:07:34.611 INFO:journalctl@ceph.mon.b.smithi196.stdout: data_block_index_type: 0 2024-09-11T08:07:34.611 INFO:journalctl@ceph.mon.b.smithi196.stdout: index_shortening: 1 2024-09-11T08:07:34.611 INFO:journalctl@ceph.mon.b.smithi196.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T08:07:34.611 INFO:journalctl@ceph.mon.b.smithi196.stdout: hash_index_allow_collision: 1 2024-09-11T08:07:34.611 INFO:journalctl@ceph.mon.b.smithi196.stdout: checksum: 1 2024-09-11T08:07:34.611 INFO:journalctl@ceph.mon.b.smithi196.stdout: no_block_cache: 0 2024-09-11T08:07:34.611 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_cache: 0x559a17c431f0 2024-09-11T08:07:34.612 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_cache_name: BinnedLRUCache 2024-09-11T08:07:34.612 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_cache_options: 2024-09-11T08:07:34.612 INFO:journalctl@ceph.mon.b.smithi196.stdout: capacity : 536870912 2024-09-11T08:07:34.612 INFO:journalctl@ceph.mon.b.smithi196.stdout: num_shard_bits : 4 2024-09-11T08:07:34.612 INFO:journalctl@ceph.mon.b.smithi196.stdout: strict_capacity_limit : 0 2024-09-11T08:07:34.612 INFO:journalctl@ceph.mon.b.smithi196.stdout: high_pri_pool_ratio: 0.000 2024-09-11T08:07:34.612 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_cache_compressed: (nil) 2024-09-11T08:07:34.612 INFO:journalctl@ceph.mon.b.smithi196.stdout: persistent_cache: (nil) 2024-09-11T08:07:34.612 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_size: 4096 2024-09-11T08:07:34.612 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_size_deviation: 10 2024-09-11T08:07:34.612 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_restart_interval: 16 2024-09-11T08:07:34.612 INFO:journalctl@ceph.mon.b.smithi196.stdout: index_block_restart_interval: 1 2024-09-11T08:07:34.612 INFO:journalctl@ceph.mon.b.smithi196.stdout: metadata_block_size: 4096 2024-09-11T08:07:34.613 INFO:journalctl@ceph.mon.b.smithi196.stdout: partition_filters: 0 2024-09-11T08:07:34.613 INFO:journalctl@ceph.mon.b.smithi196.stdout: use_delta_encoding: 1 2024-09-11T08:07:34.613 INFO:journalctl@ceph.mon.b.smithi196.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T08:07:34.613 INFO:journalctl@ceph.mon.b.smithi196.stdout: whole_key_filtering: 1 2024-09-11T08:07:34.613 INFO:journalctl@ceph.mon.b.smithi196.stdout: verify_compression: 0 2024-09-11T08:07:34.613 INFO:journalctl@ceph.mon.b.smithi196.stdout: read_amp_bytes_per_bit: 0 2024-09-11T08:07:34.614 INFO:journalctl@ceph.mon.b.smithi196.stdout: format_version: 4 2024-09-11T08:07:34.614 INFO:journalctl@ceph.mon.b.smithi196.stdout: enable_index_compression: 1 2024-09-11T08:07:34.614 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_align: 0 2024-09-11T08:07:34.614 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T08:07:34.614 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T08:07:34.614 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compression: NoCompression 2024-09-11T08:07:34.614 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T08:07:34.614 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T08:07:34.614 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T08:07:34.614 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.num_levels: 7 2024-09-11T08:07:34.614 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T08:07:34.614 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T08:07:34.614 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T08:07:34.614 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T08:07:34.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T08:07:34.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T08:07:34.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T08:07:34.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:07:34.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T08:07:34.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T08:07:34.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T08:07:34.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T08:07:34.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T08:07:34.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T08:07:34.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:07:34.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T08:07:34.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T08:07:34.616 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T08:07:34.616 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T08:07:34.616 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T08:07:34.616 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T08:07:34.616 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T08:07:34.616 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T08:07:34.616 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T08:07:34.616 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T08:07:34.616 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T08:07:34.616 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T08:07:34.616 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T08:07:34.616 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T08:07:34.616 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T08:07:34.616 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T08:07:34.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T08:07:34.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T08:07:34.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T08:07:34.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T08:07:34.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T08:07:34.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T08:07:34.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T08:07:34.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T08:07:34.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T08:07:34.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T08:07:34.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T08:07:34.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T08:07:34.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T08:07:34.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T08:07:34.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T08:07:34.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T08:07:34.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T08:07:34.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T08:07:34.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T08:07:34.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.inplace_update_support: 0 2024-09-11T08:07:34.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T08:07:34.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T08:07:34.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T08:07:34.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T08:07:34.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.bloom_locality: 0 2024-09-11T08:07:34.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.max_successive_merges: 0 2024-09-11T08:07:34.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T08:07:34.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T08:07:34.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T08:07:34.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T08:07:34.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.ttl: 2592000 2024-09-11T08:07:34.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T08:07:34.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.enable_blob_files: false 2024-09-11T08:07:34.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.min_blob_size: 0 2024-09-11T08:07:34.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T08:07:34.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T08:07:34.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T08:07:34.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T08:07:34.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T08:07:34.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-11T08:07:34.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-11T08:07:34.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726042054300812, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-11T08:07:34.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-11T08:07:34.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T08:07:34.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726042054301128, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1571, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726042054, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "63aad9d1-d5be-4873-b419-20c8139183c8", "db_session_id": "W91NS9FJVQ5DCH06HEBM"}} 2024-09-11T08:07:34.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-11T08:07:34.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726042054301443, "job": 1, "event": "recovery_finished"} 2024-09-11T08:07:34.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T08:07:34.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x559a17c76700 2024-09-11T08:07:34.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: DB pointer 0x559a17c46000 2024-09-11T08:07:34.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T08:07:34.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T08:07:34.620 INFO:journalctl@ceph.mon.b.smithi196.stdout: ** DB Stats ** 2024-09-11T08:07:34.620 INFO:journalctl@ceph.mon.b.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:07:34.621 INFO:journalctl@ceph.mon.b.smithi196.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T08:07:34.621 INFO:journalctl@ceph.mon.b.smithi196.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T08:07:34.621 INFO:journalctl@ceph.mon.b.smithi196.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:07:34.621 INFO:journalctl@ceph.mon.b.smithi196.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T08:07:34.621 INFO:journalctl@ceph.mon.b.smithi196.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T08:07:34.621 INFO:journalctl@ceph.mon.b.smithi196.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:07:34.621 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-09-11T08:07:34.621 INFO:journalctl@ceph.mon.b.smithi196.stdout: ** Compaction Stats [default] ** 2024-09-11T08:07:34.621 INFO:journalctl@ceph.mon.b.smithi196.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:07:34.621 INFO:journalctl@ceph.mon.b.smithi196.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:07:34.621 INFO:journalctl@ceph.mon.b.smithi196.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-09-11T08:07:34.621 INFO:journalctl@ceph.mon.b.smithi196.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-09-11T08:07:34.622 INFO:journalctl@ceph.mon.b.smithi196.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-09-11T08:07:34.622 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-09-11T08:07:34.622 INFO:journalctl@ceph.mon.b.smithi196.stdout: ** Compaction Stats [default] ** 2024-09-11T08:07:34.622 INFO:journalctl@ceph.mon.b.smithi196.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:07:34.622 INFO:journalctl@ceph.mon.b.smithi196.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:07:34.622 INFO:journalctl@ceph.mon.b.smithi196.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-09-11T08:07:34.622 INFO:journalctl@ceph.mon.b.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:07:34.622 INFO:journalctl@ceph.mon.b.smithi196.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:07:34.622 INFO:journalctl@ceph.mon.b.smithi196.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:07:34.622 INFO:journalctl@ceph.mon.b.smithi196.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:07:34.622 INFO:journalctl@ceph.mon.b.smithi196.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:07:34.622 INFO:journalctl@ceph.mon.b.smithi196.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:07:34.622 INFO:journalctl@ceph.mon.b.smithi196.stdout: Cumulative compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:07:34.622 INFO:journalctl@ceph.mon.b.smithi196.stdout: Interval compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:07:34.623 INFO:journalctl@ceph.mon.b.smithi196.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:07:34.623 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-09-11T08:07:34.623 INFO:journalctl@ceph.mon.b.smithi196.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:07:34.623 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-09-11T08:07:34.623 INFO:journalctl@ceph.mon.b.smithi196.stdout: ** Compaction Stats [default] ** 2024-09-11T08:07:34.623 INFO:journalctl@ceph.mon.b.smithi196.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:07:34.623 INFO:journalctl@ceph.mon.b.smithi196.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:07:34.623 INFO:journalctl@ceph.mon.b.smithi196.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-09-11T08:07:34.623 INFO:journalctl@ceph.mon.b.smithi196.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-09-11T08:07:34.623 INFO:journalctl@ceph.mon.b.smithi196.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T08:07:34.623 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-09-11T08:07:34.623 INFO:journalctl@ceph.mon.b.smithi196.stdout: ** Compaction Stats [default] ** 2024-09-11T08:07:34.623 INFO:journalctl@ceph.mon.b.smithi196.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:07:34.623 INFO:journalctl@ceph.mon.b.smithi196.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:07:34.624 INFO:journalctl@ceph.mon.b.smithi196.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-09-11T08:07:34.624 INFO:journalctl@ceph.mon.b.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:07:34.624 INFO:journalctl@ceph.mon.b.smithi196.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:07:34.624 INFO:journalctl@ceph.mon.b.smithi196.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:07:34.624 INFO:journalctl@ceph.mon.b.smithi196.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:07:34.624 INFO:journalctl@ceph.mon.b.smithi196.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:07:34.624 INFO:journalctl@ceph.mon.b.smithi196.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:07:34.624 INFO:journalctl@ceph.mon.b.smithi196.stdout: Cumulative compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:07:34.624 INFO:journalctl@ceph.mon.b.smithi196.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:07:34.624 INFO:journalctl@ceph.mon.b.smithi196.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:07:34.624 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-09-11T08:07:34.624 INFO:journalctl@ceph.mon.b.smithi196.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:07:34.624 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-11T08:07:34.625 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: using public_addr v2:172.21.15.196:0/0 -> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-09-11T08:07:34.625 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: starting mon.b rank -1 at public addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] at bind addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:07:34.625 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: mon.b@-1(???) e0 preinit fsid 566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:07:34.625 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: mon.b@-1(synchronizing).mds e1 new map 2024-09-11T08:07:34.625 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-11T08:07:34.625 INFO:journalctl@ceph.mon.b.smithi196.stdout: e1 2024-09-11T08:07:34.625 INFO:journalctl@ceph.mon.b.smithi196.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T08:07:34.625 INFO:journalctl@ceph.mon.b.smithi196.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T08:07:34.625 INFO:journalctl@ceph.mon.b.smithi196.stdout: legacy client fscid: -1 2024-09-11T08:07:34.625 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-09-11T08:07:34.625 INFO:journalctl@ceph.mon.b.smithi196.stdout: No filesystems configured 2024-09-11T08:07:34.625 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-11T08:07:34.625 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-11T08:07:34.625 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-11T08:07:34.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-11T08:07:34.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-11T08:07:34.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-11T08:07:34.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T08:07:34.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:07:34.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:07:34.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:07:34.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:34.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:34.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:34.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:34.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:34.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:34.627 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:34.627 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-11T08:07:34.627 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: Updating smithi196:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:34.627 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:34.627 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:07:34.627 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: Updating smithi196:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:07:34.627 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/2998080317' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T08:07:34.627 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:34.627 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:34.627 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:34.627 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/2998080317' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T08:07:34.627 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:07:34.627 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:34.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi071:172.21.15.71=a;smithi071:[v2:172.21.15.71:3301,v1:172.21.15.71:6790]=c;smithi196:172.21.15.196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:07:34.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: Saving service mon spec with placement smithi071:172.21.15.71=a;smithi071:[v2:172.21.15.71:3301,v1:172.21.15.71:6790]=c;smithi196:172.21.15.196=b;count:3 2024-09-11T08:07:34.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:34.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:34.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:34.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:34.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:34.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:07:34.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:34.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: Deploying daemon mon.b on smithi196 2024-09-11T08:07:34.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:34.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:34.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: from='client.? 172.21.15.196:0/2568335423' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:07:34.629 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 ceph-mon[30677]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-11T08:07:34.629 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 bash[30639]: 218d41aee37ff08574dbdf28bd06fd6169dc0cc98449cd5380691af70aa982ad 2024-09-11T08:07:34.629 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:34 smithi196 systemd[1]: Started Ceph mon.b for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:07:34.827 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:07:39.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:07:39.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: mon.a calling monitor election 2024-09-11T08:07:39.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:39.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:39.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:39.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:39.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: mon.b calling monitor election 2024-09-11T08:07:39.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:39.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:39.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:39.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:39.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T08:07:39.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: monmap e2: 2 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:07:39.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: fsmap 2024-09-11T08:07:39.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:07:39.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: mgrmap e13: y(active, since 39s) 2024-09-11T08:07:39.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: overall HEALTH_OK 2024-09-11T08:07:39.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:39.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:39.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:39.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:07:39.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:39 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:39.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:07:39.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: mon.a calling monitor election 2024-09-11T08:07:39.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:39.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:39.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:39.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:39.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: mon.b calling monitor election 2024-09-11T08:07:39.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:39.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:39.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:39.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:39.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T08:07:39.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: monmap e2: 2 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:07:39.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: fsmap 2024-09-11T08:07:39.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:07:39.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: mgrmap e13: y(active, since 39s) 2024-09-11T08:07:39.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: overall HEALTH_OK 2024-09-11T08:07:39.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:39.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:39.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:39.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:07:39.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:39 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:39.704 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T08:07:39.704 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"566367e6-7014-11ef-bcea-c7b262605968","modified":"2024-09-11T08:07:34.330012Z","created":"2024-09-11T08:06:06.240802Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T08:07:39.705 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-09-11T08:07:40.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:40 smithi196 ceph-mon[30677]: Deploying daemon mon.c on smithi071 2024-09-11T08:07:40.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:40 smithi196 ceph-mon[30677]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:40.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:40 smithi196 ceph-mon[30677]: from='client.? 172.21.15.196:0/2688590728' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:07:40.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:40 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:40.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:40 smithi071 ceph-mon[25834]: Deploying daemon mon.c on smithi071 2024-09-11T08:07:40.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:40 smithi071 ceph-mon[25834]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:40.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:40 smithi071 ceph-mon[25834]: from='client.? 172.21.15.196:0/2688590728' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:07:40.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:40 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:41.250 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T08:07:41.251 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T08:07:41.486 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:07:42.508 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 podman[39860]: 2024-09-11 08:07:42.50712071 +0000 UTC m=+0.164821351 container create e22f17c69619cff087364d7eb77b8a5cb27047399bbe2c6a3bea9f7847d397a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T08:07:42.775 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[25834]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:42.775 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 podman[39860]: 2024-09-11 08:07:42.751338284 +0000 UTC m=+0.409038934 container init e22f17c69619cff087364d7eb77b8a5cb27047399bbe2c6a3bea9f7847d397a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:07:42.775 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 podman[39860]: 2024-09-11 08:07:42.75621387 +0000 UTC m=+0.413914511 container start e22f17c69619cff087364d7eb77b8a5cb27047399bbe2c6a3bea9f7847d397a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-11T08:07:42.793 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:42 smithi196 ceph-mon[30677]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:43.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T08:07:43.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 7 2024-09-11T08:07:43.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: pidfile_write: ignore empty --pid-file 2024-09-11T08:07:43.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: load: jerasure load: lrc 2024-09-11T08:07:43.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: RocksDB version: 6.15.5 2024-09-11T08:07:43.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T08:07:43.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Compile date Sep 10 2024 2024-09-11T08:07:43.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: DB SUMMARY 2024-09-11T08:07:43.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: DB Session ID: S4638SRIVTNZZARIP36E 2024-09-11T08:07:43.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: CURRENT file: CURRENT 2024-09-11T08:07:43.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T08:07:43.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-11T08:07:43.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-11T08:07:43.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-11T08:07:43.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.error_if_exists: 0 2024-09-11T08:07:43.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.create_if_missing: 0 2024-09-11T08:07:43.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.paranoid_checks: 1 2024-09-11T08:07:43.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T08:07:43.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.env: 0x563810e43440 2024-09-11T08:07:43.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.fs: Posix File System 2024-09-11T08:07:43.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.info_log: 0x563812140ec0 2024-09-11T08:07:43.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T08:07:43.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.statistics: (nil) 2024-09-11T08:07:43.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.use_fsync: 0 2024-09-11T08:07:43.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_log_file_size: 0 2024-09-11T08:07:43.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T08:07:43.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T08:07:43.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T08:07:43.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T08:07:43.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.allow_fallocate: 1 2024-09-11T08:07:43.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T08:07:43.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T08:07:43.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.use_direct_reads: 0 2024-09-11T08:07:43.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T08:07:43.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T08:07:43.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.db_log_dir: 2024-09-11T08:07:43.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-09-11T08:07:43.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T08:07:43.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T08:07:43.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T08:07:43.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T08:07:43.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T08:07:43.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T08:07:43.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T08:07:43.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T08:07:43.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.write_buffer_manager: 0x5638121fe7b0 2024-09-11T08:07:43.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T08:07:43.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T08:07:43.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T08:07:43.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T08:07:43.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T08:07:43.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T08:07:43.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T08:07:43.200 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T08:07:43.200 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T08:07:43.200 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.unordered_write: 0 2024-09-11T08:07:43.200 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T08:07:43.200 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T08:07:43.200 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T08:07:43.200 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T08:07:43.200 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.row_cache: None 2024-09-11T08:07:43.201 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.wal_filter: None 2024-09-11T08:07:43.201 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T08:07:43.201 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T08:07:43.201 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.preserve_deletes: 0 2024-09-11T08:07:43.201 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.two_write_queues: 0 2024-09-11T08:07:43.201 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T08:07:43.201 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.atomic_flush: 0 2024-09-11T08:07:43.201 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T08:07:43.201 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T08:07:43.201 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T08:07:43.201 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.log_readahead_size: 0 2024-09-11T08:07:43.201 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T08:07:43.201 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T08:07:43.201 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T08:07:43.202 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T08:07:43.202 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T08:07:43.202 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T08:07:43.202 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_background_jobs: 2 2024-09-11T08:07:43.203 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_background_compactions: -1 2024-09-11T08:07:43.203 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_subcompactions: 1 2024-09-11T08:07:43.203 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T08:07:43.203 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T08:07:43.203 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T08:07:43.204 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T08:07:43.204 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T08:07:43.204 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T08:07:43.204 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T08:07:43.204 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T08:07:43.204 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_open_files: -1 2024-09-11T08:07:43.204 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T08:07:43.204 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T08:07:43.204 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T08:07:43.204 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T08:07:43.204 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_background_flushes: -1 2024-09-11T08:07:43.205 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Compression algorithms supported: 2024-09-11T08:07:43.205 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: kZSTD supported: 0 2024-09-11T08:07:43.205 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: kXpressCompression supported: 0 2024-09-11T08:07:43.205 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: kBZip2Compression supported: 0 2024-09-11T08:07:43.205 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T08:07:43.205 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: kLZ4Compression supported: 1 2024-09-11T08:07:43.205 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: kZlibCompression supported: 1 2024-09-11T08:07:43.205 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T08:07:43.205 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: kSnappyCompression supported: 1 2024-09-11T08:07:43.205 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T08:07:43.205 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2024-09-11T08:07:43.205 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T08:07:43.205 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T08:07:43.205 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.merge_operator: 2024-09-11T08:07:43.206 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compaction_filter: None 2024-09-11T08:07:43.206 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T08:07:43.206 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T08:07:43.206 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T08:07:43.206 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T08:07:43.206 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5638120e7a38) 2024-09-11T08:07:43.206 INFO:journalctl@ceph.mon.c.smithi071.stdout: cache_index_and_filter_blocks: 1 2024-09-11T08:07:43.206 INFO:journalctl@ceph.mon.c.smithi071.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T08:07:43.206 INFO:journalctl@ceph.mon.c.smithi071.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T08:07:43.206 INFO:journalctl@ceph.mon.c.smithi071.stdout: pin_top_level_index_and_filter: 1 2024-09-11T08:07:43.206 INFO:journalctl@ceph.mon.c.smithi071.stdout: index_type: 0 2024-09-11T08:07:43.206 INFO:journalctl@ceph.mon.c.smithi071.stdout: data_block_index_type: 0 2024-09-11T08:07:43.206 INFO:journalctl@ceph.mon.c.smithi071.stdout: index_shortening: 1 2024-09-11T08:07:43.206 INFO:journalctl@ceph.mon.c.smithi071.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T08:07:43.207 INFO:journalctl@ceph.mon.c.smithi071.stdout: hash_index_allow_collision: 1 2024-09-11T08:07:43.207 INFO:journalctl@ceph.mon.c.smithi071.stdout: checksum: 1 2024-09-11T08:07:43.207 INFO:journalctl@ceph.mon.c.smithi071.stdout: no_block_cache: 0 2024-09-11T08:07:43.207 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_cache: 0x56381211d1f0 2024-09-11T08:07:43.207 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_cache_name: BinnedLRUCache 2024-09-11T08:07:43.207 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_cache_options: 2024-09-11T08:07:43.207 INFO:journalctl@ceph.mon.c.smithi071.stdout: capacity : 536870912 2024-09-11T08:07:43.207 INFO:journalctl@ceph.mon.c.smithi071.stdout: num_shard_bits : 4 2024-09-11T08:07:43.207 INFO:journalctl@ceph.mon.c.smithi071.stdout: strict_capacity_limit : 0 2024-09-11T08:07:43.207 INFO:journalctl@ceph.mon.c.smithi071.stdout: high_pri_pool_ratio: 0.000 2024-09-11T08:07:43.207 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_cache_compressed: (nil) 2024-09-11T08:07:43.207 INFO:journalctl@ceph.mon.c.smithi071.stdout: persistent_cache: (nil) 2024-09-11T08:07:43.207 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_size: 4096 2024-09-11T08:07:43.207 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_size_deviation: 10 2024-09-11T08:07:43.208 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_restart_interval: 16 2024-09-11T08:07:43.208 INFO:journalctl@ceph.mon.c.smithi071.stdout: index_block_restart_interval: 1 2024-09-11T08:07:43.208 INFO:journalctl@ceph.mon.c.smithi071.stdout: metadata_block_size: 4096 2024-09-11T08:07:43.208 INFO:journalctl@ceph.mon.c.smithi071.stdout: partition_filters: 0 2024-09-11T08:07:43.208 INFO:journalctl@ceph.mon.c.smithi071.stdout: use_delta_encoding: 1 2024-09-11T08:07:43.208 INFO:journalctl@ceph.mon.c.smithi071.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T08:07:43.208 INFO:journalctl@ceph.mon.c.smithi071.stdout: whole_key_filtering: 1 2024-09-11T08:07:43.208 INFO:journalctl@ceph.mon.c.smithi071.stdout: verify_compression: 0 2024-09-11T08:07:43.208 INFO:journalctl@ceph.mon.c.smithi071.stdout: read_amp_bytes_per_bit: 0 2024-09-11T08:07:43.208 INFO:journalctl@ceph.mon.c.smithi071.stdout: format_version: 4 2024-09-11T08:07:43.208 INFO:journalctl@ceph.mon.c.smithi071.stdout: enable_index_compression: 1 2024-09-11T08:07:43.208 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_align: 0 2024-09-11T08:07:43.208 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T08:07:43.208 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T08:07:43.208 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compression: NoCompression 2024-09-11T08:07:43.209 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T08:07:43.209 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T08:07:43.209 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T08:07:43.209 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.num_levels: 7 2024-09-11T08:07:43.209 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T08:07:43.209 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T08:07:43.209 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T08:07:43.210 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T08:07:43.210 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T08:07:43.210 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T08:07:43.210 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T08:07:43.210 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:07:43.210 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T08:07:43.210 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T08:07:43.210 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T08:07:43.210 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T08:07:43.211 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T08:07:43.211 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T08:07:43.211 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:07:43.211 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T08:07:43.211 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T08:07:43.211 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T08:07:43.211 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T08:07:43.211 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T08:07:43.211 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T08:07:43.211 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T08:07:43.211 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T08:07:43.211 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T08:07:43.211 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T08:07:43.212 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T08:07:43.212 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T08:07:43.212 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T08:07:43.212 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T08:07:43.212 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T08:07:43.212 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T08:07:43.212 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T08:07:43.212 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T08:07:43.212 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T08:07:43.212 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T08:07:43.212 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T08:07:43.212 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T08:07:43.212 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T08:07:43.212 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T08:07:43.213 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T08:07:43.213 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T08:07:43.213 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T08:07:43.213 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T08:07:43.213 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T08:07:43.213 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T08:07:43.213 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T08:07:43.213 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T08:07:43.213 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T08:07:43.213 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T08:07:43.213 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T08:07:43.213 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.inplace_update_support: 0 2024-09-11T08:07:43.213 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T08:07:43.213 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T08:07:43.213 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T08:07:43.214 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T08:07:43.214 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.bloom_locality: 0 2024-09-11T08:07:43.214 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.max_successive_merges: 0 2024-09-11T08:07:43.214 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T08:07:43.214 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T08:07:43.214 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T08:07:43.214 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T08:07:43.214 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.ttl: 2592000 2024-09-11T08:07:43.214 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T08:07:43.214 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.enable_blob_files: false 2024-09-11T08:07:43.214 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.min_blob_size: 0 2024-09-11T08:07:43.214 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T08:07:43.214 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T08:07:43.214 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T08:07:43.215 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T08:07:43.215 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T08:07:43.215 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-11T08:07:43.215 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-11T08:07:43.215 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726042062780351, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-11T08:07:43.215 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-11T08:07:43.215 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T08:07:43.215 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726042062780661, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1696, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726042062, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "f1e06633-4987-44a3-90e5-eb32448a96e2", "db_session_id": "S4638SRIVTNZZARIP36E"}} 2024-09-11T08:07:43.215 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-11T08:07:43.215 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726042062780995, "job": 1, "event": "recovery_finished"} 2024-09-11T08:07:43.216 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T08:07:43.216 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x563812150700 2024-09-11T08:07:43.216 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: DB pointer 0x563812120000 2024-09-11T08:07:43.216 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-11T08:07:43.216 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T08:07:43.216 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T08:07:43.216 INFO:journalctl@ceph.mon.c.smithi071.stdout: ** DB Stats ** 2024-09-11T08:07:43.216 INFO:journalctl@ceph.mon.c.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:07:43.216 INFO:journalctl@ceph.mon.c.smithi071.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T08:07:43.216 INFO:journalctl@ceph.mon.c.smithi071.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T08:07:43.216 INFO:journalctl@ceph.mon.c.smithi071.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:07:43.216 INFO:journalctl@ceph.mon.c.smithi071.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T08:07:43.216 INFO:journalctl@ceph.mon.c.smithi071.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T08:07:43.216 INFO:journalctl@ceph.mon.c.smithi071.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:07:43.217 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-09-11T08:07:43.217 INFO:journalctl@ceph.mon.c.smithi071.stdout: ** Compaction Stats [default] ** 2024-09-11T08:07:43.217 INFO:journalctl@ceph.mon.c.smithi071.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:07:43.217 INFO:journalctl@ceph.mon.c.smithi071.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:07:43.217 INFO:journalctl@ceph.mon.c.smithi071.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T08:07:43.217 INFO:journalctl@ceph.mon.c.smithi071.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T08:07:43.217 INFO:journalctl@ceph.mon.c.smithi071.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T08:07:43.217 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-09-11T08:07:43.217 INFO:journalctl@ceph.mon.c.smithi071.stdout: ** Compaction Stats [default] ** 2024-09-11T08:07:43.217 INFO:journalctl@ceph.mon.c.smithi071.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:07:43.217 INFO:journalctl@ceph.mon.c.smithi071.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:07:43.217 INFO:journalctl@ceph.mon.c.smithi071.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T08:07:43.217 INFO:journalctl@ceph.mon.c.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:07:43.217 INFO:journalctl@ceph.mon.c.smithi071.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:07:43.218 INFO:journalctl@ceph.mon.c.smithi071.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:07:43.218 INFO:journalctl@ceph.mon.c.smithi071.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:07:43.218 INFO:journalctl@ceph.mon.c.smithi071.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:07:43.218 INFO:journalctl@ceph.mon.c.smithi071.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:07:43.218 INFO:journalctl@ceph.mon.c.smithi071.stdout: Cumulative compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:07:43.218 INFO:journalctl@ceph.mon.c.smithi071.stdout: Interval compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:07:43.218 INFO:journalctl@ceph.mon.c.smithi071.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:07:43.218 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-09-11T08:07:43.218 INFO:journalctl@ceph.mon.c.smithi071.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:07:43.218 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-09-11T08:07:43.218 INFO:journalctl@ceph.mon.c.smithi071.stdout: ** Compaction Stats [default] ** 2024-09-11T08:07:43.218 INFO:journalctl@ceph.mon.c.smithi071.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:07:43.218 INFO:journalctl@ceph.mon.c.smithi071.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:07:43.218 INFO:journalctl@ceph.mon.c.smithi071.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T08:07:43.219 INFO:journalctl@ceph.mon.c.smithi071.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T08:07:43.219 INFO:journalctl@ceph.mon.c.smithi071.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T08:07:43.219 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-09-11T08:07:43.219 INFO:journalctl@ceph.mon.c.smithi071.stdout: ** Compaction Stats [default] ** 2024-09-11T08:07:43.219 INFO:journalctl@ceph.mon.c.smithi071.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:07:43.219 INFO:journalctl@ceph.mon.c.smithi071.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:07:43.219 INFO:journalctl@ceph.mon.c.smithi071.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T08:07:43.219 INFO:journalctl@ceph.mon.c.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:07:43.219 INFO:journalctl@ceph.mon.c.smithi071.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:07:43.219 INFO:journalctl@ceph.mon.c.smithi071.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:07:43.219 INFO:journalctl@ceph.mon.c.smithi071.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:07:43.219 INFO:journalctl@ceph.mon.c.smithi071.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:07:43.219 INFO:journalctl@ceph.mon.c.smithi071.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:07:43.219 INFO:journalctl@ceph.mon.c.smithi071.stdout: Cumulative compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:07:43.220 INFO:journalctl@ceph.mon.c.smithi071.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:07:43.220 INFO:journalctl@ceph.mon.c.smithi071.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:07:43.220 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-09-11T08:07:43.220 INFO:journalctl@ceph.mon.c.smithi071.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:07:43.220 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: using public_addrv [v2:172.21.15.71:3301/0,v1:172.21.15.71:6790/0] 2024-09-11T08:07:43.220 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: starting mon.c rank -1 at public addrs [v2:172.21.15.71:3301/0,v1:172.21.15.71:6790/0] at bind addrs [v2:172.21.15.71:3301/0,v1:172.21.15.71:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:07:43.220 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.c@-1(???) e0 preinit fsid 566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:07:43.220 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.c@-1(synchronizing).mds e1 new map 2024-09-11T08:07:43.220 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-11T08:07:43.220 INFO:journalctl@ceph.mon.c.smithi071.stdout: e1 2024-09-11T08:07:43.220 INFO:journalctl@ceph.mon.c.smithi071.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T08:07:43.220 INFO:journalctl@ceph.mon.c.smithi071.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T08:07:43.220 INFO:journalctl@ceph.mon.c.smithi071.stdout: legacy client fscid: -1 2024-09-11T08:07:43.221 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-09-11T08:07:43.221 INFO:journalctl@ceph.mon.c.smithi071.stdout: No filesystems configured 2024-09-11T08:07:43.221 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-11T08:07:43.221 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-11T08:07:43.221 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-11T08:07:43.221 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-11T08:07:43.221 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-11T08:07:43.221 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-11T08:07:43.221 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T08:07:43.221 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:07:43.221 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:07:43.221 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:07:43.221 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:43.221 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:43.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:43.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:43.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:07:43.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:43.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:43.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-11T08:07:43.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: Updating smithi196:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:43.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:43.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:07:43.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: Updating smithi196:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:07:43.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/2998080317' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T08:07:43.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:43.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:43.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:43.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/2998080317' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T08:07:43.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:07:43.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:43.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi071:172.21.15.71=a;smithi071:[v2:172.21.15.71:3301,v1:172.21.15.71:6790]=c;smithi196:172.21.15.196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:07:43.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: Saving service mon spec with placement smithi071:172.21.15.71=a;smithi071:[v2:172.21.15.71:3301,v1:172.21.15.71:6790]=c;smithi196:172.21.15.196=b;count:3 2024-09-11T08:07:43.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:43.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:43.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:43.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:43.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:43.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:07:43.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:43.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: Deploying daemon mon.b on smithi196 2024-09-11T08:07:43.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:43.224 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:43.224 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='client.? 172.21.15.196:0/2568335423' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:07:43.224 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:07:43.224 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.a calling monitor election 2024-09-11T08:07:43.224 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:43.224 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:43.224 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:43.224 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:43.224 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.b calling monitor election 2024-09-11T08:07:43.224 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:43.224 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:43.224 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:43.224 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:43.225 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T08:07:43.225 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: monmap e2: 2 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:07:43.225 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: fsmap 2024-09-11T08:07:43.225 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:07:43.225 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mgrmap e13: y(active, since 39s) 2024-09-11T08:07:43.225 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: overall HEALTH_OK 2024-09-11T08:07:43.225 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:43.225 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:43.225 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:43.225 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:07:43.225 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:43.225 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: Deploying daemon mon.c on smithi071 2024-09-11T08:07:43.225 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:43.225 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='client.? 172.21.15.196:0/2688590728' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:07:43.225 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:43.226 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:43.226 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 ceph-mon[39897]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-11T08:07:43.226 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 bash[39860]: e22f17c69619cff087364d7eb77b8a5cb27047399bbe2c6a3bea9f7847d397a8 2024-09-11T08:07:43.226 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:42 smithi071 systemd[1]: Started Ceph mon.c for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:07:48.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:07:48.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: mon.a calling monitor election 2024-09-11T08:07:48.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: mon.b calling monitor election 2024-09-11T08:07:48.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:48.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:48.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: mon.c calling monitor election 2024-09-11T08:07:48.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:48.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T08:07:48.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: monmap e3: 3 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.71:3301/0,v1:172.21.15.71:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:07:48.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: fsmap 2024-09-11T08:07:48.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:07:48.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: mgrmap e13: y(active, since 48s) 2024-09-11T08:07:48.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: overall HEALTH_OK 2024-09-11T08:07:48.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:48.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:48.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:48.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:47 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:48.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:07:48.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: mon.a calling monitor election 2024-09-11T08:07:48.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: mon.b calling monitor election 2024-09-11T08:07:48.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:48.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:48.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: mon.c calling monitor election 2024-09-11T08:07:48.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:48.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T08:07:48.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: monmap e3: 3 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.71:3301/0,v1:172.21.15.71:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:07:48.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: fsmap 2024-09-11T08:07:48.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:07:48.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: mgrmap e13: y(active, since 48s) 2024-09-11T08:07:48.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: overall HEALTH_OK 2024-09-11T08:07:48.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:48.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:48.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:48.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:48.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:07:48.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: mon.a calling monitor election 2024-09-11T08:07:48.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: mon.b calling monitor election 2024-09-11T08:07:48.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:07:48.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:48.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: mon.c calling monitor election 2024-09-11T08:07:48.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:48.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:48.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T08:07:48.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: monmap e3: 3 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.71:3301/0,v1:172.21.15.71:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:07:48.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: fsmap 2024-09-11T08:07:48.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:07:48.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: mgrmap e13: y(active, since 48s) 2024-09-11T08:07:48.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: overall HEALTH_OK 2024-09-11T08:07:48.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:48.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:48.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:48.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:47 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:49.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:48 smithi196 ceph-mon[30677]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:49.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:48 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:49.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:48 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:49.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:48 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:49.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:48 smithi071 ceph-mon[39897]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:49.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:48 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:49.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:48 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:49.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:48 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:49.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:48 smithi071 ceph-mon[25834]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:49.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:48 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:49.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:48 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:07:49.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:48 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:49.241 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T08:07:49.241 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"fsid":"566367e6-7014-11ef-bcea-c7b262605968","modified":"2024-09-11T08:07:42.802998Z","created":"2024-09-11T08:06:06.240802Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3301","nonce":0},{"type":"v1","addr":"172.21.15.71:6790","nonce":0}]},"addr":"172.21.15.71:6790/0","public_addr":"172.21.15.71:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-11T08:07:49.241 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 3 2024-09-11T08:07:49.879 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T08:07:49.879 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T08:07:50.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:50.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:50.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-11T08:07:50.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-11T08:07:50.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: from='client.? 172.21.15.196:0/2107040809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:07:50.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:07:50.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:07:50.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:50.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:49 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:50.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:50.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-11T08:07:50.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-11T08:07:50.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: from='client.? 172.21.15.196:0/2107040809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:07:50.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:07:50.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:07:50.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:50.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:50.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:50.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-11T08:07:50.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-11T08:07:50.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: from='client.? 172.21.15.196:0/2107040809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:07:50.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:07:50.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:07:50.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:50.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:49 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:50.890 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:51.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[25834]: Updating smithi071:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:51.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[25834]: Updating smithi196:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:51.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[25834]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T08:07:51.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[25834]: Reconfiguring daemon mon.a on smithi071 2024-09-11T08:07:51.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[25834]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:51.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:51.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:51.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:07:51.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:07:51.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:51.173 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[39897]: Updating smithi071:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:51.173 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[39897]: Updating smithi196:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:51.173 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[39897]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T08:07:51.173 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[39897]: Reconfiguring daemon mon.a on smithi071 2024-09-11T08:07:51.173 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[39897]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:51.173 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:51.173 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:51.173 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:07:51.173 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:07:51.173 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:50 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:51.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:50 smithi196 ceph-mon[30677]: Updating smithi071:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:51.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:50 smithi196 ceph-mon[30677]: Updating smithi196:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:07:51.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:50 smithi196 ceph-mon[30677]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T08:07:51.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:50 smithi196 ceph-mon[30677]: Reconfiguring daemon mon.a on smithi071 2024-09-11T08:07:51.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:50 smithi196 ceph-mon[30677]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:51.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:50 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:51.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:50 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:51.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:50 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:07:51.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:50 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:07:51.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:50 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:52.119 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:51 smithi071 ceph-mon[25834]: Reconfiguring mon.c (monmap changed)... 2024-09-11T08:07:52.119 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:51 smithi071 ceph-mon[25834]: Reconfiguring daemon mon.c on smithi071 2024-09-11T08:07:52.119 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:51 smithi071 ceph-mon[39897]: Reconfiguring mon.c (monmap changed)... 2024-09-11T08:07:52.120 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:51 smithi071 ceph-mon[39897]: Reconfiguring daemon mon.c on smithi071 2024-09-11T08:07:52.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:51 smithi196 ceph-mon[30677]: Reconfiguring mon.c (monmap changed)... 2024-09-11T08:07:52.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:51 smithi196 ceph-mon[30677]: Reconfiguring daemon mon.c on smithi071 2024-09-11T08:07:53.050 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:52 smithi071 ceph-mon[25834]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:53.050 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:52 smithi071 ceph-mon[39897]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:53.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:52 smithi196 ceph-mon[30677]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:53.297 INFO:teuthology.orchestra.run.smithi071.stdout:# minimal ceph.conf for 566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:07:53.298 INFO:teuthology.orchestra.run.smithi071.stdout:[global] 2024-09-11T08:07:53.298 INFO:teuthology.orchestra.run.smithi071.stdout: fsid = 566367e6-7014-11ef-bcea-c7b262605968 2024-09-11T08:07:53.298 INFO:teuthology.orchestra.run.smithi071.stdout: mon_host = [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] [v2:172.21.15.71:3301/0,v1:172.21.15.71:6790/0] 2024-09-11T08:07:54.122 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:53 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/3163410253' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:54.122 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:53 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:54.122 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:53 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:54.122 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:53 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:07:54.122 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:53 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:07:54.122 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:53 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:54.123 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:53 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3163410253' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:54.123 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:53 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:54.123 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:53 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:54.123 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:53 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:07:54.123 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:53 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:07:54.123 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:53 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:54.146 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T08:07:54.146 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T08:07:54.146 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T08:07:54.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:53 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/3163410253' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:54.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:53 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:54.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:53 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:54.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:53 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:07:54.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:53 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:07:54.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:53 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:54.173 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T08:07:54.174 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:07:54.239 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T08:07:54.240 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T08:07:54.268 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T08:07:54.268 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:07:54.334 INFO:tasks.cephadm:Adding mgr.y on smithi071 2024-09-11T08:07:54.334 INFO:tasks.cephadm:Adding mgr.x on smithi196 2024-09-11T08:07:54.335 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi071=y;smithi196=x' 2024-09-11T08:07:54.943 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:07:55.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:54 smithi196 ceph-mon[30677]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:55.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:54 smithi196 ceph-mon[30677]: Reconfiguring mon.b (monmap changed)... 2024-09-11T08:07:55.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:54 smithi196 ceph-mon[30677]: Reconfiguring daemon mon.b on smithi196 2024-09-11T08:07:55.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:54 smithi071 ceph-mon[39897]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:55.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:54 smithi071 ceph-mon[39897]: Reconfiguring mon.b (monmap changed)... 2024-09-11T08:07:55.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:54 smithi071 ceph-mon[39897]: Reconfiguring daemon mon.b on smithi196 2024-09-11T08:07:55.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:54 smithi071 ceph-mon[25834]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:55.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:54 smithi071 ceph-mon[25834]: Reconfiguring mon.b (monmap changed)... 2024-09-11T08:07:55.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:54 smithi071 ceph-mon[25834]: Reconfiguring daemon mon.b on smithi196 2024-09-11T08:07:56.093 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:55 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:56.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:55 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:56.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:55 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:56.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:55 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:56.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:55 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:56.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:55 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:56.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:55 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:56.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:55 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:56.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:55 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:56.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:55 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:56.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:55 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:56.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:55 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:56.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:55 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:56.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:55 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:56.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:55 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:56.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:55 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:56.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:55 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:56.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:55 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:56.580 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mgr update... 2024-09-11T08:07:57.099 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:56 smithi196 ceph-mon[30677]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:57.099 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:57.099 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:57.099 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:57.099 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:57.099 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:57.099 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:07:57.100 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T08:07:57.100 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:07:57.100 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:57.121 DEBUG:teuthology.orchestra.run.smithi196:mgr.x> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T08:07:57.123 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T08:07:57.123 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T08:07:57.123 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T08:07:57.143 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-11T08:07:57.144 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/nvme4n1 2024-09-11T08:07:57.202 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/nvme4n1 2024-09-11T08:07:57.202 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-11T08:07:57.202 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 1040 Links: 1 Device type: 103,8 2024-09-11T08:07:57.202 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-11T08:07:57.202 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-11T08:07:57.202 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-11 08:07:12.895394766 +0000 2024-09-11T08:07:57.202 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-11 08:03:30.413277055 +0000 2024-09-11T08:07:57.202 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-11 08:03:30.413277055 +0000 2024-09-11T08:07:57.202 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-11 08:03:30.409277125 +0000 2024-09-11T08:07:57.203 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-11T08:07:57.264 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[25834]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:57.265 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:57.265 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:57.265 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:57.265 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:57.265 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:57.265 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:07:57.265 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T08:07:57.265 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:07:57.265 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:57.265 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[39897]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:57.265 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:57.266 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:57.266 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:57.266 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:57.266 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:07:57.266 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:07:57.266 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T08:07:57.266 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:07:57.266 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:57.267 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-11T08:07:57.267 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-11T08:07:57.267 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000206117 s, 2.5 MB/s 2024-09-11T08:07:57.268 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-11T08:07:57.325 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/nvme3n1 2024-09-11T08:07:57.382 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/nvme3n1 2024-09-11T08:07:57.382 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-11T08:07:57.382 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 1027 Links: 1 Device type: 103,6 2024-09-11T08:07:57.382 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-11T08:07:57.382 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-11T08:07:57.382 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-11 08:07:12.885394941 +0000 2024-09-11T08:07:57.382 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-11 08:03:30.252279866 +0000 2024-09-11T08:07:57.382 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-11 08:03:30.252279866 +0000 2024-09-11T08:07:57.382 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-11 08:03:30.248279935 +0000 2024-09-11T08:07:57.383 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-11T08:07:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-11T08:07:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-11T08:07:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000219614 s, 2.3 MB/s 2024-09-11T08:07:57.447 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-11T08:07:57.504 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/nvme2n1 2024-09-11T08:07:57.559 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/nvme2n1 2024-09-11T08:07:57.559 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-11T08:07:57.559 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 1015 Links: 1 Device type: 103,4 2024-09-11T08:07:57.559 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-11T08:07:57.559 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-11T08:07:57.560 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-11 08:07:12.874395133 +0000 2024-09-11T08:07:57.560 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-11 08:03:30.112282309 +0000 2024-09-11T08:07:57.560 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-11 08:03:30.112282309 +0000 2024-09-11T08:07:57.560 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-11 08:03:30.107282396 +0000 2024-09-11T08:07:57.560 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-11T08:07:57.625 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-11T08:07:57.625 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-11T08:07:57.626 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000224124 s, 2.3 MB/s 2024-09-11T08:07:57.627 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-11T08:07:57.683 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/nvme1n1 2024-09-11T08:07:57.738 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/nvme1n1 2024-09-11T08:07:57.738 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-11T08:07:57.738 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-11T08:07:57.738 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-11T08:07:57.739 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-11T08:07:57.739 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-11 08:07:12.863395325 +0000 2024-09-11T08:07:57.739 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-11 08:03:29.982284578 +0000 2024-09-11T08:07:57.739 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-11 08:03:29.982284578 +0000 2024-09-11T08:07:57.739 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-11 08:03:29.976284683 +0000 2024-09-11T08:07:57.739 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-11T08:07:57.802 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-11T08:07:57.803 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-11T08:07:57.803 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000211347 s, 2.4 MB/s 2024-09-11T08:07:57.804 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-11T08:07:57.860 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T08:07:57.860 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T08:07:57.877 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-11T08:07:57.877 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/nvme4n1 2024-09-11T08:07:57.932 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/nvme4n1 2024-09-11T08:07:57.933 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-11T08:07:57.933 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 1040 Links: 1 Device type: 103,8 2024-09-11T08:07:57.933 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-11T08:07:57.933 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-11T08:07:57.933 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-11 08:07:24.122212363 +0000 2024-09-11T08:07:57.933 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-11 08:03:32.001260842 +0000 2024-09-11T08:07:57.933 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-11 08:03:32.001260842 +0000 2024-09-11T08:07:57.933 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-11 08:03:31.997260930 +0000 2024-09-11T08:07:57.933 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-11T08:07:57.997 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-11T08:07:57.997 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-11T08:07:57.997 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000229956 s, 2.2 MB/s 2024-09-11T08:07:57.999 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-11T08:07:58.055 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/nvme3n1 2024-09-11T08:07:58.112 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/nvme3n1 2024-09-11T08:07:58.113 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-11T08:07:58.113 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 1028 Links: 1 Device type: 103,6 2024-09-11T08:07:58.113 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-11T08:07:58.113 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-11T08:07:58.113 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-11 08:07:24.112212580 +0000 2024-09-11T08:07:58.113 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-11 08:03:31.859263958 +0000 2024-09-11T08:07:58.113 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-11 08:03:31.859263958 +0000 2024-09-11T08:07:58.113 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-11 08:03:31.854264068 +0000 2024-09-11T08:07:58.113 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-11T08:07:58.176 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-11T08:07:58.176 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-11T08:07:58.176 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000234811 s, 2.2 MB/s 2024-09-11T08:07:58.178 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-11T08:07:58.233 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/nvme2n1 2024-09-11T08:07:58.289 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/nvme2n1 2024-09-11T08:07:58.289 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-11T08:07:58.290 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 1015 Links: 1 Device type: 103,4 2024-09-11T08:07:58.290 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-11T08:07:58.290 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-11T08:07:58.290 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-11 08:07:24.101212819 +0000 2024-09-11T08:07:58.290 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-11 08:03:31.719267032 +0000 2024-09-11T08:07:58.290 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-11 08:03:31.719267032 +0000 2024-09-11T08:07:58.290 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-11 08:03:31.715267119 +0000 2024-09-11T08:07:58.290 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-11T08:07:58.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:57 smithi196 ceph-mon[30677]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi071=y;smithi196=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:07:58.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:57 smithi196 ceph-mon[30677]: Saving service mgr spec with placement smithi071=y;smithi196=x;count:2 2024-09-11T08:07:58.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:57 smithi196 ceph-mon[30677]: Deploying daemon mgr.x on smithi196 2024-09-11T08:07:58.326 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:57 smithi071 ceph-mon[39897]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi071=y;smithi196=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:07:58.326 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:57 smithi071 ceph-mon[39897]: Saving service mgr spec with placement smithi071=y;smithi196=x;count:2 2024-09-11T08:07:58.326 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:57 smithi071 ceph-mon[39897]: Deploying daemon mgr.x on smithi196 2024-09-11T08:07:58.326 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:57 smithi071 ceph-mon[25834]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi071=y;smithi196=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:07:58.326 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:57 smithi071 ceph-mon[25834]: Saving service mgr spec with placement smithi071=y;smithi196=x;count:2 2024-09-11T08:07:58.327 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:57 smithi071 ceph-mon[25834]: Deploying daemon mgr.x on smithi196 2024-09-11T08:07:58.496 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-11T08:07:58.496 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-11T08:07:58.496 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000201806 s, 2.5 MB/s 2024-09-11T08:07:58.497 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-11T08:07:58.513 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/nvme1n1 2024-09-11T08:07:58.574 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/nvme1n1 2024-09-11T08:07:58.575 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-11T08:07:58.575 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-11T08:07:58.575 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-11T08:07:58.575 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-11T08:07:58.575 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-11 08:07:24.091213036 +0000 2024-09-11T08:07:58.575 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-11 08:03:31.588269907 +0000 2024-09-11T08:07:58.575 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-11 08:03:31.588269907 +0000 2024-09-11T08:07:58.575 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-11 08:03:31.583270016 +0000 2024-09-11T08:07:58.575 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-11T08:07:58.687 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-11T08:07:58.687 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-11T08:07:58.687 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000288185 s, 1.8 MB/s 2024-09-11T08:07:58.688 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-11T08:07:58.704 INFO:tasks.cephadm:Deploying osd.0 on smithi071 with /dev/nvme1n1... 2024-09-11T08:07:58.704 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-11T08:07:58.898 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:07:58 smithi196 systemd[1]: Starting Ceph mgr.x for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:07:58.936 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:07:59.056 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:07:58 smithi071 ceph-mon[25834]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:59.056 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:07:58 smithi071 ceph-mon[39897]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:59.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:07:58 smithi196 ceph-mon[30677]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:07:59.181 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:07:58 smithi196 podman[32746]: 2024-09-11 08:07:58.898028108 +0000 UTC m=+0.094748764 container create 2cc3e7f0e1d3ced938feb1cb198c0b64d20ba5d57814883be4efb0d7ee83733f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T08:07:59.182 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:07:58 smithi196 podman[32746]: 2024-09-11 08:07:58.818895649 +0000 UTC m=+0.015616313 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:07:59.182 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:07:59 smithi196 podman[32746]: 2024-09-11 08:07:59.100527519 +0000 UTC m=+0.297248179 container init 2cc3e7f0e1d3ced938feb1cb198c0b64d20ba5d57814883be4efb0d7ee83733f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:07:59.182 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:07:59 smithi196 podman[32746]: 2024-09-11 08:07:59.105377566 +0000 UTC m=+0.302098222 container start 2cc3e7f0e1d3ced938feb1cb198c0b64d20ba5d57814883be4efb0d7ee83733f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T08:07:59.182 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:07:59 smithi196 ceph-mgr[32783]: -- 172.21.15.196:0/2470111966 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d0fcf0c340 con 0x55d0fcf18c00 2024-09-11T08:07:59.477 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:07:59 smithi196 bash[32746]: 2cc3e7f0e1d3ced938feb1cb198c0b64d20ba5d57814883be4efb0d7ee83733f 2024-09-11T08:07:59.477 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:07:59 smithi196 systemd[1]: Started Ceph mgr.x for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:07:59.477 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:07:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:07:59.220+0000 7fab1ec9aec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:07:59.477 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:07:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:07:59.278+0000 7fab1ec9aec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:07:59.478 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:07:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:07:59.346+0000 7fab1ec9aec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:07:59.478 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:07:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:07:59.476+0000 7fab1ec9aec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:07:59.909 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:07:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:07:59.633+0000 7fab1ec9aec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:07:59.910 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:07:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:07:59.908+0000 7fab1ec9aec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:08:00.181 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:07:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:07:59.973+0000 7fab1ec9aec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:08:00.181 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:00 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:00.145+0000 7fab1ec9aec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:08:00.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:00 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:00.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:00 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:00.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:00 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:00.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:00 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:00.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:00 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:00.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:00 smithi071 ceph-mon[25834]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:00.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:00 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:00.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:00 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:00.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:00 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:00.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:00 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:00.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:00 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:00.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:00 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:00.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:00 smithi071 ceph-mon[39897]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:00.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:00 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:00.632 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/nvme1n1 2024-09-11T08:08:00.632 INFO:teuthology.orchestra.run.smithi071.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-11T08:08:00.636 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-11T08:08:00.651 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-09-11T08:08:00.651 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-09-11T08:08:00.651 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130915 s, 801 MB/s 2024-09-11T08:08:00.651 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-09-11T08:08:00.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:00 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:00.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:00 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:00.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:00 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:00.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:00 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:00.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:00 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:00.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:00 smithi196 ceph-mon[30677]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:00.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:00 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:01.330 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi071:/dev/nvme1n1 2024-09-11T08:08:01.431 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:00 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:00.995+0000 7fab1ec9aec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:08:01.431 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:01 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:01.057+0000 7fab1ec9aec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:08:01.613 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:01.613 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:01.613 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:01.613 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:01.613 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:01.613 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[25834]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T08:08:01.613 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:08:01.613 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:08:01.614 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:01.614 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[25834]: Reconfiguring daemon mgr.y on smithi071 2024-09-11T08:08:01.614 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:01.614 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:01.614 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:01.614 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:01.614 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:01.614 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[39897]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T08:08:01.614 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:08:01.614 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:08:01.614 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:01.614 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:01 smithi071 ceph-mon[39897]: Reconfiguring daemon mgr.y on smithi071 2024-09-11T08:08:01.768 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:01 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:01.769 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:01 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:01.769 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:01 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:01.769 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:01 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:01.769 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:01 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:01.769 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:01 smithi196 ceph-mon[30677]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T08:08:01.769 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:01 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:08:01.769 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:01 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:08:01.769 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:01 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:01.769 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:01 smithi196 ceph-mon[30677]: Reconfiguring daemon mgr.y on smithi071 2024-09-11T08:08:02.058 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:01 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:01.767+0000 7fab1ec9aec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:08:02.058 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:01 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:01.878+0000 7fab1ec9aec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:08:02.058 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:01 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:01.998+0000 7fab1ec9aec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:08:02.059 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:02 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:02.057+0000 7fab1ec9aec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:08:02.077 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:08:02.431 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:02 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:02.182+0000 7fab1ec9aec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:08:02.431 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:02 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:02.248+0000 7fab1ec9aec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:08:02.431 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:02 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:02.304+0000 7fab1ec9aec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:08:02.701 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:02 smithi196 ceph-mon[30677]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:02.701 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:02 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:02.437+0000 7fab1ec9aec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:08:02.701 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:02 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:02.500+0000 7fab1ec9aec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:08:02.702 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:02 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:02.564+0000 7fab1ec9aec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:08:02.702 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:02 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:02.644+0000 7fab1ec9aec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:08:02.702 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:02 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:02.700+0000 7fab1ec9aec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:08:02.727 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:02 smithi071 ceph-mon[25834]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:02.727 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:02 smithi071 ceph-mon[39897]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:03.181 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:02 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:02.756+0000 7fab1ec9aec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:08:03.794 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:08:03 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:08:03.499+0000 7fab1ec9aec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:08:03.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:03.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:03.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:03.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:03.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:03.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:03.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[25834]: Standby manager daemon x started 2024-09-11T08:08:03.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[25834]: from='mgr.? 172.21.15.196:0/3469428241' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:08:03.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[25834]: from='mgr.? 172.21.15.196:0/3469428241' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:08:03.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[25834]: from='mgr.? 172.21.15.196:0/3469428241' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:08:03.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[25834]: from='mgr.? 172.21.15.196:0/3469428241' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:08:03.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:03.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:03.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:03.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:03.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:03.947 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:03.947 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[39897]: Standby manager daemon x started 2024-09-11T08:08:03.947 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[39897]: from='mgr.? 172.21.15.196:0/3469428241' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:08:03.947 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[39897]: from='mgr.? 172.21.15.196:0/3469428241' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:08:03.947 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[39897]: from='mgr.? 172.21.15.196:0/3469428241' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:08:03.947 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:03 smithi071 ceph-mon[39897]: from='mgr.? 172.21.15.196:0/3469428241' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:08:04.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:03 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:04.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:03 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:04.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:03 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:04.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:03 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:04.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:03 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:04.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:03 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:04.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:03 smithi196 ceph-mon[30677]: Standby manager daemon x started 2024-09-11T08:08:04.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:03 smithi196 ceph-mon[30677]: from='mgr.? 172.21.15.196:0/3469428241' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:08:04.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:03 smithi196 ceph-mon[30677]: from='mgr.? 172.21.15.196:0/3469428241' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:08:04.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:03 smithi196 ceph-mon[30677]: from='mgr.? 172.21.15.196:0/3469428241' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:08:04.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:03 smithi196 ceph-mon[30677]: from='mgr.? 172.21.15.196:0/3469428241' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:08:04.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:04 smithi071 ceph-mon[39897]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:04.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:04 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:08:04.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:04 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:08:04.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:04 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:04.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:04 smithi071 ceph-mon[39897]: mgrmap e14: y(active, since 64s), standbys: x 2024-09-11T08:08:04.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:04 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:08:04.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:04 smithi071 ceph-mon[25834]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:04.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:04 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:08:04.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:04 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:08:04.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:04 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:04.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:04 smithi071 ceph-mon[25834]: mgrmap e14: y(active, since 64s), standbys: x 2024-09-11T08:08:04.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:04 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:08:05.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:04 smithi196 ceph-mon[30677]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:05.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:04 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:08:05.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:04 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:08:05.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:04 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:05.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:04 smithi196 ceph-mon[30677]: mgrmap e14: y(active, since 64s), standbys: x 2024-09-11T08:08:05.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:04 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:08:06.016 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:05 smithi071 ceph-mon[25834]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:08:06.016 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:05 smithi071 ceph-mon[39897]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:08:06.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:05 smithi196 ceph-mon[30677]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:08:07.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:06 smithi196 ceph-mon[30677]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:07.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:06 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/654047320' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b831aea3-cb55-49cc-ad5c-3c340ec2f249"}]: dispatch 2024-09-11T08:08:07.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:06 smithi196 ceph-mon[30677]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b831aea3-cb55-49cc-ad5c-3c340ec2f249"}]: dispatch 2024-09-11T08:08:07.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:06 smithi196 ceph-mon[30677]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b831aea3-cb55-49cc-ad5c-3c340ec2f249"}]': finished 2024-09-11T08:08:07.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:06 smithi196 ceph-mon[30677]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T08:08:07.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:06 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:07.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:06 smithi071 ceph-mon[25834]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:07.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:06 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/654047320' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b831aea3-cb55-49cc-ad5c-3c340ec2f249"}]: dispatch 2024-09-11T08:08:07.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:06 smithi071 ceph-mon[25834]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b831aea3-cb55-49cc-ad5c-3c340ec2f249"}]: dispatch 2024-09-11T08:08:07.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:06 smithi071 ceph-mon[25834]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b831aea3-cb55-49cc-ad5c-3c340ec2f249"}]': finished 2024-09-11T08:08:07.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:06 smithi071 ceph-mon[25834]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T08:08:07.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:06 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:07.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:06 smithi071 ceph-mon[39897]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:07.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:06 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/654047320' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b831aea3-cb55-49cc-ad5c-3c340ec2f249"}]: dispatch 2024-09-11T08:08:07.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:06 smithi071 ceph-mon[39897]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b831aea3-cb55-49cc-ad5c-3c340ec2f249"}]: dispatch 2024-09-11T08:08:07.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:06 smithi071 ceph-mon[39897]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b831aea3-cb55-49cc-ad5c-3c340ec2f249"}]': finished 2024-09-11T08:08:07.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:06 smithi071 ceph-mon[39897]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T08:08:07.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:06 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:09.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:08 smithi196 ceph-mon[30677]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:09.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:08 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/3161289065' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:08:09.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:08 smithi071 ceph-mon[39897]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:09.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:08 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/3161289065' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:08:09.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:08 smithi071 ceph-mon[25834]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:09.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:08 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3161289065' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:08:11.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:10 smithi071 ceph-mon[25834]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:11.168 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:10 smithi071 ceph-mon[39897]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:11.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:10 smithi196 ceph-mon[30677]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:13.135 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:12 smithi071 ceph-mon[25834]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:13.135 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:12 smithi071 ceph-mon[39897]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:12 smithi196 ceph-mon[30677]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:14.097 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:13 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:14.097 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:13 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:14.098 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:13 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:14.098 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:13 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:14.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:13 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T08:08:14.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:13 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:14.834 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:14 smithi071 ceph-mon[39897]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:14.834 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:14 smithi071 ceph-mon[39897]: Deploying daemon osd.0 on smithi071 2024-09-11T08:08:14.834 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:14 smithi071 ceph-mon[25834]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:14.834 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:14 smithi071 ceph-mon[25834]: Deploying daemon osd.0 on smithi071 2024-09-11T08:08:15.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:14 smithi196 ceph-mon[30677]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:15.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:14 smithi196 ceph-mon[30677]: Deploying daemon osd.0 on smithi071 2024-09-11T08:08:16.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:16 smithi071 ceph-mon[25834]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:16.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:16 smithi071 ceph-mon[39897]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:17.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:16 smithi196 ceph-mon[30677]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:19.114 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:18 smithi071 ceph-mon[25834]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:19.114 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:18 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:19.114 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:18 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:19.114 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:18 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:19.114 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:18 smithi071 ceph-mon[39897]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:19.114 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:18 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:19.114 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:18 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:19.114 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:18 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:19.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:18 smithi196 ceph-mon[30677]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:19.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:18 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:19.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:18 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:19.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:18 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:21.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:20 smithi196 ceph-mon[30677]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:21.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:20 smithi071 ceph-mon[25834]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:21.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:20 smithi071 ceph-mon[39897]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:21.556 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 0 on host 'smithi071' 2024-09-11T08:08:22.190 DEBUG:teuthology.orchestra.run.smithi071:osd.0> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T08:08:22.193 INFO:tasks.cephadm:Deploying osd.1 on smithi071 with /dev/nvme2n1... 2024-09-11T08:08:22.193 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme2n1 2024-09-11T08:08:22.492 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:08:22.885 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:22 smithi071 ceph-mon[25834]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:22.885 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:22 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:22.885 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:22 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:22.885 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:22 smithi071 ceph-mon[25834]: from='osd.0 [v2:172.21.15.71:6802/1906336816,v1:172.21.15.71:6803/1906336816]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:08:22.885 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:22 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:22.885 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:22 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:22.886 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:22 smithi071 ceph-mon[39897]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:22.886 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:22 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:22.886 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:22 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:22.886 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:22 smithi071 ceph-mon[39897]: from='osd.0 [v2:172.21.15.71:6802/1906336816,v1:172.21.15.71:6803/1906336816]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:08:22.886 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:22 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:22.886 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:22 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:22.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:22 smithi196 ceph-mon[30677]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:22.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:22 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:22.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:22 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:22.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:22 smithi196 ceph-mon[30677]: from='osd.0 [v2:172.21.15.71:6802/1906336816,v1:172.21.15.71:6803/1906336816]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:08:22.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:22 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:22.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:22 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:23.647 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:23 smithi071 ceph-mon[39897]: from='osd.0 [v2:172.21.15.71:6802/1906336816,v1:172.21.15.71:6803/1906336816]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T08:08:23.647 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:23 smithi071 ceph-mon[39897]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T08:08:23.647 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:23 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:23.647 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:23 smithi071 ceph-mon[39897]: from='osd.0 [v2:172.21.15.71:6802/1906336816,v1:172.21.15.71:6803/1906336816]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T08:08:23.648 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:23 smithi071 ceph-mon[25834]: from='osd.0 [v2:172.21.15.71:6802/1906336816,v1:172.21.15.71:6803/1906336816]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T08:08:23.648 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:23 smithi071 ceph-mon[25834]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T08:08:23.648 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:23 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:23.648 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:23 smithi071 ceph-mon[25834]: from='osd.0 [v2:172.21.15.71:6802/1906336816,v1:172.21.15.71:6803/1906336816]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T08:08:23.648 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:08:23 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-0[44270]: 2024-09-11T08:08:23.564+0000 7fb17a750640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T08:08:23.648 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:08:23 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-0[44270]: 2024-09-11T08:08:23.567+0000 7fb17758b640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:08:23.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:23 smithi196 ceph-mon[30677]: from='osd.0 [v2:172.21.15.71:6802/1906336816,v1:172.21.15.71:6803/1906336816]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T08:08:23.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:23 smithi196 ceph-mon[30677]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T08:08:23.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:23 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:23.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:23 smithi196 ceph-mon[30677]: from='osd.0 [v2:172.21.15.71:6802/1906336816,v1:172.21.15.71:6803/1906336816]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T08:08:24.568 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:24 smithi071 ceph-mon[25834]: purged_snaps scrub starts 2024-09-11T08:08:24.598 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/nvme2n1 2024-09-11T08:08:24.598 INFO:teuthology.orchestra.run.smithi071.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-11T08:08:24.603 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme2n1 bs=1M count=10 conv=fsync 2024-09-11T08:08:24.620 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-09-11T08:08:24.620 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-09-11T08:08:24.620 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0148036 s, 708 MB/s 2024-09-11T08:08:24.620 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-09-11T08:08:24.874 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:24 smithi071 ceph-mon[25834]: purged_snaps scrub ok 2024-09-11T08:08:24.965 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:24 smithi071 ceph-mon[25834]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:24.965 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:24 smithi071 ceph-mon[25834]: from='osd.0 [v2:172.21.15.71:6802/1906336816,v1:172.21.15.71:6803/1906336816]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T08:08:24.965 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:24 smithi071 ceph-mon[25834]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T08:08:24.966 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:24 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:24.966 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:24 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:24.966 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:24 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:24.966 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:24 smithi071 ceph-mon[39897]: purged_snaps scrub starts 2024-09-11T08:08:24.966 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:24 smithi071 ceph-mon[39897]: purged_snaps scrub ok 2024-09-11T08:08:24.966 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:24 smithi071 ceph-mon[39897]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:24.966 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:24 smithi071 ceph-mon[39897]: from='osd.0 [v2:172.21.15.71:6802/1906336816,v1:172.21.15.71:6803/1906336816]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T08:08:24.966 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:24 smithi071 ceph-mon[39897]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T08:08:24.966 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:24 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:24.966 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:24 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:24.966 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:24 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:24.967 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:24 smithi196 ceph-mon[30677]: purged_snaps scrub starts 2024-09-11T08:08:24.967 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:24 smithi196 ceph-mon[30677]: purged_snaps scrub ok 2024-09-11T08:08:24.967 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:24 smithi196 ceph-mon[30677]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:24.967 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:24 smithi196 ceph-mon[30677]: from='osd.0 [v2:172.21.15.71:6802/1906336816,v1:172.21.15.71:6803/1906336816]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T08:08:24.967 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:24 smithi196 ceph-mon[30677]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T08:08:24.967 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:24 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:24.967 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:24 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:24.967 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:24 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:25.762 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:25 smithi071 ceph-mon[25834]: osd.0 [v2:172.21.15.71:6802/1906336816,v1:172.21.15.71:6803/1906336816] boot 2024-09-11T08:08:25.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:25 smithi071 ceph-mon[25834]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T08:08:25.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:25 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:25.763 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:25 smithi071 ceph-mon[39897]: osd.0 [v2:172.21.15.71:6802/1906336816,v1:172.21.15.71:6803/1906336816] boot 2024-09-11T08:08:25.763 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:25 smithi071 ceph-mon[39897]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T08:08:25.763 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:25 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:25.786 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi071:/dev/nvme2n1 2024-09-11T08:08:25.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:25 smithi196 ceph-mon[30677]: osd.0 [v2:172.21.15.71:6802/1906336816,v1:172.21.15.71:6803/1906336816] boot 2024-09-11T08:08:25.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:25 smithi196 ceph-mon[30677]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T08:08:25.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:25 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:08:26.231 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:08:26.668 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:26 smithi071 ceph-mon[25834]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:26.669 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:26 smithi071 ceph-mon[39897]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:26.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:26 smithi196 ceph-mon[30677]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:08:27.616 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:27 smithi071 ceph-mon[39897]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T08:08:27.616 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:27 smithi071 ceph-mon[25834]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T08:08:27.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:27 smithi196 ceph-mon[30677]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T08:08:28.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:28 smithi196 ceph-mon[30677]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:28.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:28 smithi196 ceph-mon[30677]: Detected new or changed devices on smithi071 2024-09-11T08:08:28.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:28 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:28.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:28 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:28.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:28 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:08:28.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:28 smithi196 ceph-mon[30677]: Adjusting osd_memory_target on smithi071 to 16064M 2024-09-11T08:08:28.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:28 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:28.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:28 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:28.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:28 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:28.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:28 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:28.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:28 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:28.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:28 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:28.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:28 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:28.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:28 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:28.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[25834]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:28.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[25834]: Detected new or changed devices on smithi071 2024-09-11T08:08:28.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:28.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:28.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:08:28.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[25834]: Adjusting osd_memory_target on smithi071 to 16064M 2024-09-11T08:08:28.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:28.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:28.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:28.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:28.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:28.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:28.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:28.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:28.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[39897]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:28.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[39897]: Detected new or changed devices on smithi071 2024-09-11T08:08:28.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:28.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:28.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:08:28.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[39897]: Adjusting osd_memory_target on smithi071 to 16064M 2024-09-11T08:08:28.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:28.947 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:28.947 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:28.947 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:28.947 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:28.947 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:28.947 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:28.947 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:28 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:29.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:29 smithi071 ceph-mon[39897]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:08:29.697 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:29 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:08:29.697 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:29 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:08:29.697 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:29 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:29.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:29 smithi071 ceph-mon[25834]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:08:29.698 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:29 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:08:29.698 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:29 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:08:29.698 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:29 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:29.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:29 smithi196 ceph-mon[30677]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:08:29.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:29 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:08:29.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:29 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:08:29.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:29 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:30.749 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:30 smithi071 ceph-mon[25834]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:30.749 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:30 smithi071 ceph-mon[39897]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:30.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:30 smithi196 ceph-mon[30677]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:31.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:31 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/2506415925' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "168eac60-ee21-42ca-85ea-844e9073441d"}]: dispatch 2024-09-11T08:08:31.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:31 smithi196 ceph-mon[30677]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "168eac60-ee21-42ca-85ea-844e9073441d"}]: dispatch 2024-09-11T08:08:31.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:31 smithi196 ceph-mon[30677]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "168eac60-ee21-42ca-85ea-844e9073441d"}]': finished 2024-09-11T08:08:31.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:31 smithi196 ceph-mon[30677]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T08:08:31.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:31 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:08:31.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:31 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/2506415925' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "168eac60-ee21-42ca-85ea-844e9073441d"}]: dispatch 2024-09-11T08:08:31.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:31 smithi071 ceph-mon[39897]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "168eac60-ee21-42ca-85ea-844e9073441d"}]: dispatch 2024-09-11T08:08:31.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:31 smithi071 ceph-mon[39897]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "168eac60-ee21-42ca-85ea-844e9073441d"}]': finished 2024-09-11T08:08:31.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:31 smithi071 ceph-mon[39897]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T08:08:31.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:31 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:08:31.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:31 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2506415925' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "168eac60-ee21-42ca-85ea-844e9073441d"}]: dispatch 2024-09-11T08:08:31.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:31 smithi071 ceph-mon[25834]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "168eac60-ee21-42ca-85ea-844e9073441d"}]: dispatch 2024-09-11T08:08:31.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:31 smithi071 ceph-mon[25834]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "168eac60-ee21-42ca-85ea-844e9073441d"}]': finished 2024-09-11T08:08:31.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:31 smithi071 ceph-mon[25834]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T08:08:31.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:31 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:08:32.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:32 smithi196 ceph-mon[30677]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:32.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:32 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/2724880154' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:08:32.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:32 smithi071 ceph-mon[25834]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:32.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:32 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2724880154' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:08:32.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:32 smithi071 ceph-mon[39897]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:32.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:32 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/2724880154' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:08:34.925 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:34 smithi071 ceph-mon[25834]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:34.925 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:34 smithi071 ceph-mon[39897]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:34.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:34 smithi196 ceph-mon[30677]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:36.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:36 smithi071 ceph-mon[25834]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:36.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:36 smithi071 ceph-mon[39897]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:36.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:36 smithi196 ceph-mon[30677]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:38.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:38 smithi071 ceph-mon[25834]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:38.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:38 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T08:08:38.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:38 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:38.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:38 smithi071 ceph-mon[25834]: Deploying daemon osd.1 on smithi071 2024-09-11T08:08:38.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:38 smithi071 ceph-mon[39897]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:38.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:38 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T08:08:38.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:38 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:38.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:38 smithi071 ceph-mon[39897]: Deploying daemon osd.1 on smithi071 2024-09-11T08:08:38.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:38 smithi196 ceph-mon[30677]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:38.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:38 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T08:08:38.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:38 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:38.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:38 smithi196 ceph-mon[30677]: Deploying daemon osd.1 on smithi071 2024-09-11T08:08:40.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:40 smithi196 ceph-mon[30677]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:40.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:40 smithi071 ceph-mon[25834]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:40.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:40 smithi071 ceph-mon[39897]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:42.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:42 smithi071 ceph-mon[25834]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:42.679 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:42 smithi071 ceph-mon[39897]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:42.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:42 smithi196 ceph-mon[30677]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:43.918 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:43 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:43.919 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:43 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:43.919 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:43 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:43.919 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:43 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:43.919 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:43 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:43.919 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:43 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:43.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:43 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:43.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:43 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:43.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:43 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:44.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:44 smithi196 ceph-mon[30677]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:44.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:44 smithi071 ceph-mon[25834]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:44.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:44 smithi071 ceph-mon[39897]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:46.301 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 1 on host 'smithi071' 2024-09-11T08:08:46.853 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:46 smithi071 ceph-mon[25834]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:46.854 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:46 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:46.854 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:46 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:46.854 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:46 smithi071 ceph-mon[25834]: from='osd.1 [v2:172.21.15.71:6810/1800678301,v1:172.21.15.71:6811/1800678301]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:08:46.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:46 smithi071 ceph-mon[39897]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:46.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:46 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:46.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:46 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:46.854 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:46 smithi071 ceph-mon[39897]: from='osd.1 [v2:172.21.15.71:6810/1800678301,v1:172.21.15.71:6811/1800678301]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:08:46.878 DEBUG:teuthology.orchestra.run.smithi071:osd.1> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T08:08:46.881 INFO:tasks.cephadm:Deploying osd.2 on smithi071 with /dev/nvme3n1... 2024-09-11T08:08:46.881 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme3n1 2024-09-11T08:08:46.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:46 smithi196 ceph-mon[30677]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:46.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:46 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:46.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:46 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:46.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:46 smithi196 ceph-mon[30677]: from='osd.1 [v2:172.21.15.71:6810/1800678301,v1:172.21.15.71:6811/1800678301]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:08:47.168 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:08:48.273 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:47 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:48.273 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:47 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:48.273 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:47 smithi071 ceph-mon[25834]: from='osd.1 [v2:172.21.15.71:6810/1800678301,v1:172.21.15.71:6811/1800678301]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T08:08:48.273 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:47 smithi071 ceph-mon[25834]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T08:08:48.273 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:47 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:08:48.273 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:47 smithi071 ceph-mon[25834]: from='osd.1 [v2:172.21.15.71:6810/1800678301,v1:172.21.15.71:6811/1800678301]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T08:08:48.273 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:47 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:48.273 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:47 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:48.274 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:47 smithi071 ceph-mon[39897]: from='osd.1 [v2:172.21.15.71:6810/1800678301,v1:172.21.15.71:6811/1800678301]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T08:08:48.274 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:47 smithi071 ceph-mon[39897]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T08:08:48.274 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:47 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:08:48.274 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:47 smithi071 ceph-mon[39897]: from='osd.1 [v2:172.21.15.71:6810/1800678301,v1:172.21.15.71:6811/1800678301]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T08:08:48.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:47 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:48.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:47 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:48.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:47 smithi196 ceph-mon[30677]: from='osd.1 [v2:172.21.15.71:6810/1800678301,v1:172.21.15.71:6811/1800678301]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T08:08:48.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:47 smithi196 ceph-mon[30677]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T08:08:48.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:47 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:08:48.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:47 smithi196 ceph-mon[30677]: from='osd.1 [v2:172.21.15.71:6810/1800678301,v1:172.21.15.71:6811/1800678301]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T08:08:48.525 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:08:48 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-1[51856]: 2024-09-11T08:08:48.308+0000 7f13e261e640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T08:08:48.525 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:08:48 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-1[51856]: 2024-09-11T08:08:48.311+0000 7f13dd455640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:08:49.193 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:48 smithi071 ceph-mon[39897]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:49.193 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:48 smithi071 ceph-mon[39897]: from='osd.1 [v2:172.21.15.71:6810/1800678301,v1:172.21.15.71:6811/1800678301]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T08:08:49.194 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:48 smithi071 ceph-mon[39897]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T08:08:49.194 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:48 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:08:49.194 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:48 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:08:49.194 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:48 smithi071 ceph-mon[25834]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:49.194 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:48 smithi071 ceph-mon[25834]: from='osd.1 [v2:172.21.15.71:6810/1800678301,v1:172.21.15.71:6811/1800678301]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T08:08:49.194 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:48 smithi071 ceph-mon[25834]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T08:08:49.194 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:48 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:08:49.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:48 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:08:49.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:48 smithi196 ceph-mon[30677]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:49.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:48 smithi196 ceph-mon[30677]: from='osd.1 [v2:172.21.15.71:6810/1800678301,v1:172.21.15.71:6811/1800678301]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T08:08:49.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:48 smithi196 ceph-mon[30677]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T08:08:49.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:48 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:08:49.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:48 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:08:50.343 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/nvme3n1 2024-09-11T08:08:50.344 INFO:teuthology.orchestra.run.smithi071.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-11T08:08:50.348 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme3n1 bs=1M count=10 conv=fsync 2024-09-11T08:08:50.364 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-09-11T08:08:50.364 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-09-11T08:08:50.365 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0139303 s, 753 MB/s 2024-09-11T08:08:50.365 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-09-11T08:08:50.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:50 smithi196 ceph-mon[30677]: purged_snaps scrub starts 2024-09-11T08:08:50.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:50 smithi196 ceph-mon[30677]: purged_snaps scrub ok 2024-09-11T08:08:50.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:50 smithi196 ceph-mon[30677]: osd.1 [v2:172.21.15.71:6810/1800678301,v1:172.21.15.71:6811/1800678301] boot 2024-09-11T08:08:50.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:50 smithi196 ceph-mon[30677]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T08:08:50.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:50 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:08:50.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:50 smithi196 ceph-mon[30677]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:50.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:50 smithi071 ceph-mon[25834]: purged_snaps scrub starts 2024-09-11T08:08:50.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:50 smithi071 ceph-mon[25834]: purged_snaps scrub ok 2024-09-11T08:08:50.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:50 smithi071 ceph-mon[25834]: osd.1 [v2:172.21.15.71:6810/1800678301,v1:172.21.15.71:6811/1800678301] boot 2024-09-11T08:08:50.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:50 smithi071 ceph-mon[25834]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T08:08:50.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:50 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:08:50.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:50 smithi071 ceph-mon[25834]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:50.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:50 smithi071 ceph-mon[39897]: purged_snaps scrub starts 2024-09-11T08:08:50.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:50 smithi071 ceph-mon[39897]: purged_snaps scrub ok 2024-09-11T08:08:50.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:50 smithi071 ceph-mon[39897]: osd.1 [v2:172.21.15.71:6810/1800678301,v1:172.21.15.71:6811/1800678301] boot 2024-09-11T08:08:50.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:50 smithi071 ceph-mon[39897]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T08:08:50.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:50 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:08:50.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:50 smithi071 ceph-mon[39897]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:08:51.020 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi071:/dev/nvme3n1 2024-09-11T08:08:51.641 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:51 smithi071 ceph-mon[39897]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T08:08:51.641 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:51 smithi071 ceph-mon[25834]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T08:08:51.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:51 smithi196 ceph-mon[30677]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T08:08:52.030 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:08:53.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:52 smithi196 ceph-mon[30677]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:08:53.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:52 smithi196 ceph-mon[30677]: Detected new or changed devices on smithi071 2024-09-11T08:08:53.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:53.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:53.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:08:53.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:08:53.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:52 smithi196 ceph-mon[30677]: Adjusting osd_memory_target on smithi071 to 8032M 2024-09-11T08:08:53.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:53.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:53.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:53.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:53.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:53.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:53.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:53.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:53.183 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[25834]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:08:53.183 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[25834]: Detected new or changed devices on smithi071 2024-09-11T08:08:53.183 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:53.183 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:53.183 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:08:53.183 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:08:53.183 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[25834]: Adjusting osd_memory_target on smithi071 to 8032M 2024-09-11T08:08:53.183 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:53.184 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:53.184 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:53.184 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:53.184 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:53.184 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:53.184 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:53.184 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:53.184 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[39897]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:08:53.184 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[39897]: Detected new or changed devices on smithi071 2024-09-11T08:08:53.184 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:53.184 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:53.185 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:08:53.185 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:08:53.185 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[39897]: Adjusting osd_memory_target on smithi071 to 8032M 2024-09-11T08:08:53.185 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:53.185 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:53.185 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:53.185 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:53.185 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:53.185 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:53.185 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:53.185 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:08:54.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:53 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:08:54.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:53 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:08:54.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:53 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:54.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:53 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:08:54.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:53 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:08:54.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:53 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:54.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:53 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:08:54.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:53 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:08:54.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:53 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:55.176 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:54 smithi071 ceph-mon[39897]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:08:55.176 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:54 smithi071 ceph-mon[39897]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:08:55.176 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:54 smithi071 ceph-mon[25834]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:08:55.176 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:54 smithi071 ceph-mon[25834]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:08:55.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:54 smithi196 ceph-mon[30677]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:08:55.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:54 smithi196 ceph-mon[30677]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:08:57.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:56 smithi071 ceph-mon[25834]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:08:57.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:56 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/1347233547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c3bbd0e-1a4a-42ad-8cd8-29c663141966"}]: dispatch 2024-09-11T08:08:57.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:56 smithi071 ceph-mon[25834]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c3bbd0e-1a4a-42ad-8cd8-29c663141966"}]: dispatch 2024-09-11T08:08:57.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:56 smithi071 ceph-mon[25834]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c3bbd0e-1a4a-42ad-8cd8-29c663141966"}]': finished 2024-09-11T08:08:57.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:56 smithi071 ceph-mon[25834]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T08:08:57.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:08:57.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:56 smithi071 ceph-mon[39897]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:08:57.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:56 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/1347233547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c3bbd0e-1a4a-42ad-8cd8-29c663141966"}]: dispatch 2024-09-11T08:08:57.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:56 smithi071 ceph-mon[39897]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c3bbd0e-1a4a-42ad-8cd8-29c663141966"}]: dispatch 2024-09-11T08:08:57.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:56 smithi071 ceph-mon[39897]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c3bbd0e-1a4a-42ad-8cd8-29c663141966"}]': finished 2024-09-11T08:08:57.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:56 smithi071 ceph-mon[39897]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T08:08:57.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:08:57.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:56 smithi196 ceph-mon[30677]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:08:57.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:56 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/1347233547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c3bbd0e-1a4a-42ad-8cd8-29c663141966"}]: dispatch 2024-09-11T08:08:57.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:56 smithi196 ceph-mon[30677]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c3bbd0e-1a4a-42ad-8cd8-29c663141966"}]: dispatch 2024-09-11T08:08:57.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:56 smithi196 ceph-mon[30677]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c3bbd0e-1a4a-42ad-8cd8-29c663141966"}]': finished 2024-09-11T08:08:57.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:56 smithi196 ceph-mon[30677]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T08:08:57.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:08:58.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:57 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2268260399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:08:58.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:57 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/2268260399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:08:58.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:57 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/2268260399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:08:59.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:08:58 smithi071 ceph-mon[25834]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:08:59.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:08:58 smithi071 ceph-mon[39897]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:08:59.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:08:58 smithi196 ceph-mon[30677]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:01.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:00 smithi071 ceph-mon[25834]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:01.760 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:00 smithi071 ceph-mon[39897]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:01.760 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:00 smithi196 ceph-mon[30677]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:03.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:02 smithi071 ceph-mon[25834]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:03.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:02 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T08:09:03.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:02 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:03.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:02 smithi071 ceph-mon[39897]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:03.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:02 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T08:09:03.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:02 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:03.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:02 smithi196 ceph-mon[30677]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:03.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:02 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T08:09:03.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:02 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:04.229 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:03 smithi071 ceph-mon[39897]: Deploying daemon osd.2 on smithi071 2024-09-11T08:09:04.229 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:03 smithi071 ceph-mon[25834]: Deploying daemon osd.2 on smithi071 2024-09-11T08:09:04.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:03 smithi196 ceph-mon[30677]: Deploying daemon osd.2 on smithi071 2024-09-11T08:09:05.110 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:04 smithi071 ceph-mon[25834]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:05.111 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:04 smithi071 ceph-mon[39897]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:05.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:04 smithi196 ceph-mon[30677]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:07.244 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:06 smithi071 ceph-mon[25834]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:07.245 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:06 smithi071 ceph-mon[39897]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:07.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:06 smithi196 ceph-mon[30677]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:08.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:07 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:08.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:07 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:08.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:07 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:08.167 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:07 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:08.167 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:07 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:08.168 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:07 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:08.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:07 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:08.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:07 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:08.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:07 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:09.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:08 smithi071 ceph-mon[25834]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:09.171 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:08 smithi071 ceph-mon[39897]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:09.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:08 smithi196 ceph-mon[30677]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:11.016 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:10 smithi071 ceph-mon[39897]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:11.016 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:10 smithi071 ceph-mon[39897]: from='osd.2 [v2:172.21.15.71:6818/1556215465,v1:172.21.15.71:6819/1556215465]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:09:11.017 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:10 smithi071 ceph-mon[25834]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:11.017 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:10 smithi071 ceph-mon[25834]: from='osd.2 [v2:172.21.15.71:6818/1556215465,v1:172.21.15.71:6819/1556215465]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:09:11.048 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 2 on host 'smithi071' 2024-09-11T08:09:11.233 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:10 smithi196 ceph-mon[30677]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:11.234 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:10 smithi196 ceph-mon[30677]: from='osd.2 [v2:172.21.15.71:6818/1556215465,v1:172.21.15.71:6819/1556215465]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:09:11.782 DEBUG:teuthology.orchestra.run.smithi071:osd.2> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T08:09:11.785 INFO:tasks.cephadm:Deploying osd.3 on smithi071 with /dev/nvme4n1... 2024-09-11T08:09:11.785 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme4n1 2024-09-11T08:09:12.080 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:09:12.195 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-2[60429]: 2024-09-11T08:09:11.975+0000 7f66c5ab8640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T08:09:12.195 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-2[60429]: 2024-09-11T08:09:11.981+0000 7f66c10f0640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:09:12.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[25834]: from='osd.2 [v2:172.21.15.71:6818/1556215465,v1:172.21.15.71:6819/1556215465]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T08:09:12.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[25834]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T08:09:12.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:09:12.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[25834]: from='osd.2 [v2:172.21.15.71:6818/1556215465,v1:172.21.15.71:6819/1556215465]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T08:09:12.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:12.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:12.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:12.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:12.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[25834]: from='osd.2 [v2:172.21.15.71:6818/1556215465,v1:172.21.15.71:6819/1556215465]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T08:09:12.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[25834]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T08:09:12.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:09:12.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[39897]: from='osd.2 [v2:172.21.15.71:6818/1556215465,v1:172.21.15.71:6819/1556215465]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T08:09:12.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[39897]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T08:09:12.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:09:12.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[39897]: from='osd.2 [v2:172.21.15.71:6818/1556215465,v1:172.21.15.71:6819/1556215465]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T08:09:12.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:12.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:12.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:12.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:12.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[39897]: from='osd.2 [v2:172.21.15.71:6818/1556215465,v1:172.21.15.71:6819/1556215465]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T08:09:12.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[39897]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T08:09:12.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:11 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:09:12.406 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:11 smithi196 ceph-mon[30677]: from='osd.2 [v2:172.21.15.71:6818/1556215465,v1:172.21.15.71:6819/1556215465]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T08:09:12.406 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:11 smithi196 ceph-mon[30677]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T08:09:12.406 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:11 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:09:12.406 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:11 smithi196 ceph-mon[30677]: from='osd.2 [v2:172.21.15.71:6818/1556215465,v1:172.21.15.71:6819/1556215465]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T08:09:12.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:11 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:12.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:11 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:12.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:11 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:12.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:11 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:12.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:11 smithi196 ceph-mon[30677]: from='osd.2 [v2:172.21.15.71:6818/1556215465,v1:172.21.15.71:6819/1556215465]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T08:09:12.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:11 smithi196 ceph-mon[30677]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T08:09:12.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:11 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:09:13.227 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:12 smithi071 ceph-mon[39897]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:13.228 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:12 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:09:13.228 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:12 smithi071 ceph-mon[25834]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:13.228 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:12 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:09:13.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:12 smithi196 ceph-mon[30677]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:13.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:12 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:09:13.942 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/nvme4n1 2024-09-11T08:09:13.942 INFO:teuthology.orchestra.run.smithi071.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-11T08:09:13.946 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-09-11T08:09:13.961 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-09-11T08:09:13.961 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-09-11T08:09:13.961 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131622 s, 797 MB/s 2024-09-11T08:09:13.961 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-09-11T08:09:14.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:13 smithi071 ceph-mon[25834]: purged_snaps scrub starts 2024-09-11T08:09:14.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:13 smithi071 ceph-mon[25834]: purged_snaps scrub ok 2024-09-11T08:09:14.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:13 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:09:14.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:13 smithi071 ceph-mon[25834]: osd.2 [v2:172.21.15.71:6818/1556215465,v1:172.21.15.71:6819/1556215465] boot 2024-09-11T08:09:14.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:13 smithi071 ceph-mon[25834]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T08:09:14.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:13 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:09:14.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:13 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T08:09:14.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:13 smithi071 ceph-mon[39897]: purged_snaps scrub starts 2024-09-11T08:09:14.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:13 smithi071 ceph-mon[39897]: purged_snaps scrub ok 2024-09-11T08:09:14.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:13 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:09:14.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:13 smithi071 ceph-mon[39897]: osd.2 [v2:172.21.15.71:6818/1556215465,v1:172.21.15.71:6819/1556215465] boot 2024-09-11T08:09:14.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:13 smithi071 ceph-mon[39897]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T08:09:14.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:13 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:09:14.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:13 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T08:09:14.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:13 smithi196 ceph-mon[30677]: purged_snaps scrub starts 2024-09-11T08:09:14.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:13 smithi196 ceph-mon[30677]: purged_snaps scrub ok 2024-09-11T08:09:14.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:13 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:09:14.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:13 smithi196 ceph-mon[30677]: osd.2 [v2:172.21.15.71:6818/1556215465,v1:172.21.15.71:6819/1556215465] boot 2024-09-11T08:09:14.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:13 smithi196 ceph-mon[30677]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T08:09:14.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:13 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:09:14.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:13 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T08:09:15.009 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi071:/dev/nvme4n1 2024-09-11T08:09:15.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:15 smithi196 ceph-mon[30677]: pgmap v75: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:15.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:15 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T08:09:15.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:15 smithi196 ceph-mon[30677]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T08:09:15.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:15 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:15.441 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:15 smithi071 ceph-mon[39897]: pgmap v75: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:15.441 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:15 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T08:09:15.441 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:15 smithi071 ceph-mon[39897]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T08:09:15.441 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:15 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:15.442 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:14 smithi071 ceph-mon[25834]: pgmap v75: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:09:15.442 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:14 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T08:09:15.442 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:14 smithi071 ceph-mon[25834]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T08:09:15.442 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:15 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:16.005 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:16 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:09:16.005 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:16 smithi071 ceph-mon[39897]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T08:09:16.005 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:16 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:09:16.005 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:16 smithi071 ceph-mon[25834]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T08:09:16.006 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:09:15 smithi071 sudo[62089]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2024-09-11T08:09:16.006 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:09:15 smithi071 sudo[62089]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:09:16.006 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:09:15 smithi071 sudo[62089]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:09:16.006 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:09:15 smithi071 sudo[62089]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:09:16.023 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:09:16.293 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:09:16 smithi071 sudo[62137]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme1n1 2024-09-11T08:09:16.294 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:09:16 smithi071 sudo[62137]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:09:16.294 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:09:16 smithi071 sudo[62137]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:09:16.294 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:09:16 smithi071 sudo[62137]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:09:16.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:16 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:09:16.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:16 smithi196 ceph-mon[30677]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T08:09:16.557 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:09:16 smithi071 sudo[62199]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2024-09-11T08:09:16.557 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:09:16 smithi071 sudo[62199]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:09:16.557 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:09:16 smithi071 sudo[62199]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:09:16.557 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:09:16 smithi071 sudo[62199]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:09:16.924 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:09:16 smithi071 sudo[64217]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme2n1 2024-09-11T08:09:16.925 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:09:16 smithi071 sudo[64217]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:09:16.925 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:09:16 smithi071 sudo[64217]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:09:16.925 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:09:16 smithi071 sudo[64217]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:09:17.187 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:17 smithi071 ceph-mon[39897]: pgmap v78: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:17.187 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:17 smithi071 ceph-mon[39897]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T08:09:17.187 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:17 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:17.187 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:17 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:17.187 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:17 smithi071 ceph-mon[25834]: pgmap v78: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:17.187 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:17 smithi071 ceph-mon[25834]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T08:09:17.187 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:17 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:17.187 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:17 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:17.188 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:09:16 smithi071 sudo[67194]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme3n1 2024-09-11T08:09:17.188 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:09:16 smithi071 sudo[67194]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:09:17.188 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:09:16 smithi071 sudo[67194]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:09:17.188 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:09:16 smithi071 sudo[67194]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:09:17.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:17 smithi196 ceph-mon[30677]: pgmap v78: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:17.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:17 smithi196 ceph-mon[30677]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T08:09:17.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:17 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:17.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:17 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:17.444 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:09:17 smithi071 sudo[67325]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme3n1 2024-09-11T08:09:17.445 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:09:17 smithi071 sudo[67325]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:09:17.445 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:09:17 smithi071 sudo[67325]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:09:17.445 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:09:17 smithi071 sudo[67325]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:09:17.720 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:17 smithi071 sudo[67350]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:09:17.721 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:17 smithi071 sudo[67350]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:09:17.721 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:17 smithi071 sudo[67350]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:09:17.721 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:17 smithi071 sudo[67350]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:09:17.995 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:17 smithi071 sudo[67377]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:09:17.995 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:17 smithi071 sudo[67377]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:09:17.996 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:17 smithi071 sudo[67377]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:09:17.996 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:17 smithi071 sudo[67377]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:09:18.238 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:17 smithi196 sudo[33387]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:09:18.238 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:17 smithi196 sudo[33387]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:09:18.238 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:17 smithi196 sudo[33387]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:09:18.238 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 sudo[33387]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:09:18.252 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:18.252 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[25834]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:09:18.252 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[25834]: pgmap v80: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:18.252 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[25834]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:09:18.252 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:09:18.252 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:09:18.252 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:09:18.252 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:18.252 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:09:18.252 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:09:18.253 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:09:18.253 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[25834]: Detected new or changed devices on smithi071 2024-09-11T08:09:18.519 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:18.520 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[39897]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:09:18.520 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[39897]: pgmap v80: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:18.520 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[39897]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:09:18.520 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:09:18.520 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:09:18.520 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:09:18.520 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:18.520 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:09:18.520 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:09:18.520 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:09:18.520 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 ceph-mon[39897]: Detected new or changed devices on smithi071 2024-09-11T08:09:18.520 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 sudo[67476]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:09:18.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 sudo[33390]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:09:18.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 sudo[33390]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:09:18.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 sudo[33390]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:09:18.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 sudo[33390]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:09:18.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:18.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 ceph-mon[30677]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:09:18.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 ceph-mon[30677]: pgmap v80: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:18.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 ceph-mon[30677]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:09:18.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:09:18.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:09:18.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:09:18.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:18.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:09:18.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:09:18.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:09:18.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:18 smithi196 ceph-mon[30677]: Detected new or changed devices on smithi071 2024-09-11T08:09:18.778 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 sudo[67476]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:09:18.779 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 sudo[67476]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:09:18.779 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 sudo[67476]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:09:19.056 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 sudo[67534]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:09:19.056 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 sudo[67534]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:09:19.056 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 sudo[67534]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:09:19.056 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:18 smithi071 sudo[67534]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:09:19.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:09:19.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:09:19.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T08:09:19.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:19.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:19.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:09:19.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:09:19.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:09:19.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:09:19.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:19.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: Adjusting osd_memory_target on smithi071 to 5354M 2024-09-11T08:09:19.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:09:19.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:09:19.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:09:19.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:19.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:09:19.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:19.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:19.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:19.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:19.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:19.448 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:09:19.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:09:19.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T08:09:19.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:19.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.449 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:19.449 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:09:19.449 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:09:19.449 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:09:19.449 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:09:19.449 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:19.449 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: Adjusting osd_memory_target on smithi071 to 5354M 2024-09-11T08:09:19.449 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:09:19.450 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:09:19.450 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:09:19.450 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:19.450 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:09:19.450 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.450 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:19.450 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:19.450 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.450 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:19.450 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:19.450 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:19.450 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:19 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:09:19.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:09:19.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T08:09:19.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:19.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:19.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:09:19.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:09:19.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:09:19.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:09:19.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:19.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: Adjusting osd_memory_target on smithi071 to 5354M 2024-09-11T08:09:19.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:09:19.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:09:19.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:09:19.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:19.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:09:19.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:19.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:19.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:19.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:19.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:19.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:19.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:19 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:20.398 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:20 smithi071 ceph-mon[25834]: pgmap v81: 1 pgs: 1 creating+activating; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:20.398 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:20 smithi071 ceph-mon[39897]: pgmap v81: 1 pgs: 1 creating+activating; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:20.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:20 smithi196 ceph-mon[30677]: pgmap v81: 1 pgs: 1 creating+activating; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:21.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:21 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/2481729342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "418fdf0b-3c97-4e70-9696-0c879202d3b7"}]: dispatch 2024-09-11T08:09:21.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:21 smithi196 ceph-mon[30677]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "418fdf0b-3c97-4e70-9696-0c879202d3b7"}]: dispatch 2024-09-11T08:09:21.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:21 smithi196 ceph-mon[30677]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "418fdf0b-3c97-4e70-9696-0c879202d3b7"}]': finished 2024-09-11T08:09:21.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:21 smithi196 ceph-mon[30677]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T08:09:21.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:21 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:21.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:21 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2481729342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "418fdf0b-3c97-4e70-9696-0c879202d3b7"}]: dispatch 2024-09-11T08:09:21.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:21 smithi071 ceph-mon[25834]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "418fdf0b-3c97-4e70-9696-0c879202d3b7"}]: dispatch 2024-09-11T08:09:21.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:21 smithi071 ceph-mon[25834]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "418fdf0b-3c97-4e70-9696-0c879202d3b7"}]': finished 2024-09-11T08:09:21.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:21 smithi071 ceph-mon[25834]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T08:09:21.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:21.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:21 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/2481729342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "418fdf0b-3c97-4e70-9696-0c879202d3b7"}]: dispatch 2024-09-11T08:09:21.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:21 smithi071 ceph-mon[39897]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "418fdf0b-3c97-4e70-9696-0c879202d3b7"}]: dispatch 2024-09-11T08:09:21.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:21 smithi071 ceph-mon[39897]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "418fdf0b-3c97-4e70-9696-0c879202d3b7"}]': finished 2024-09-11T08:09:21.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:21 smithi071 ceph-mon[39897]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T08:09:21.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:21 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:22.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:22 smithi071 ceph-mon[25834]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:22.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:22 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/120348875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:09:22.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:22 smithi071 ceph-mon[39897]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:22.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:22 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/120348875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:09:22.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:22 smithi196 ceph-mon[30677]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:22.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:22 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/120348875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:09:24.924 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:24 smithi071 ceph-mon[39897]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:24.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:24 smithi071 ceph-mon[25834]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:24.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:24 smithi196 ceph-mon[30677]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:26.566 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:26 smithi071 ceph-mon[39897]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:26.567 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:26 smithi071 ceph-mon[25834]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:26.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:26 smithi196 ceph-mon[30677]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:27.868 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:27 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T08:09:27.868 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:27 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:27.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:27 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T08:09:27.868 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:27 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:27.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:27 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T08:09:27.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:27 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:28.828 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:28 smithi071 ceph-mon[39897]: Deploying daemon osd.3 on smithi071 2024-09-11T08:09:28.829 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:28 smithi071 ceph-mon[39897]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:28.829 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:28 smithi071 ceph-mon[25834]: Deploying daemon osd.3 on smithi071 2024-09-11T08:09:28.829 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:28 smithi071 ceph-mon[25834]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:28.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:28 smithi196 ceph-mon[30677]: Deploying daemon osd.3 on smithi071 2024-09-11T08:09:28.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:28 smithi196 ceph-mon[30677]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:30.818 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:30 smithi071 ceph-mon[25834]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:30.819 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:30 smithi071 ceph-mon[39897]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:30.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:30 smithi196 ceph-mon[30677]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:32.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:32 smithi071 ceph-mon[39897]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:32.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:32 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:32.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:32 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:32.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:32 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:32.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:32 smithi071 ceph-mon[25834]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:32 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:32 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:32 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:32.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:32 smithi196 ceph-mon[30677]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:32.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:32 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:32.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:32 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:32.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:32 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:34.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:34 smithi196 ceph-mon[30677]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:34.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:34 smithi071 ceph-mon[25834]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:34.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:34 smithi071 ceph-mon[39897]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:35.619 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 3 on host 'smithi071' 2024-09-11T08:09:36.257 DEBUG:teuthology.orchestra.run.smithi071:osd.3> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T08:09:36.260 INFO:tasks.cephadm:Deploying osd.4 on smithi196 with /dev/nvme1n1... 2024-09-11T08:09:36.260 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-11T08:09:36.506 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:09:36.918 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:36 smithi071 ceph-mon[25834]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:36.919 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:36 smithi071 ceph-mon[25834]: from='osd.3 [v2:172.21.15.71:6826/2205658375,v1:172.21.15.71:6827/2205658375]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:09:36.919 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:36 smithi071 ceph-mon[25834]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:09:36.919 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:36 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:36.919 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:36 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:36.919 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:36 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:36.919 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:36 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:36.919 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:36 smithi071 ceph-mon[39897]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:36.919 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:36 smithi071 ceph-mon[39897]: from='osd.3 [v2:172.21.15.71:6826/2205658375,v1:172.21.15.71:6827/2205658375]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:09:36.919 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:36 smithi071 ceph-mon[39897]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:09:36.919 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:36 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:36.920 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:36 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:36.920 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:36 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:36.920 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:36 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:36.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:36 smithi196 ceph-mon[30677]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:36.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:36 smithi196 ceph-mon[30677]: from='osd.3 [v2:172.21.15.71:6826/2205658375,v1:172.21.15.71:6827/2205658375]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:09:36.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:36 smithi196 ceph-mon[30677]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:09:36.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:36 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:36.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:36 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:36.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:36 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:36.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:36 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:37.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:37 smithi196 ceph-mon[30677]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T08:09:37.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:37 smithi196 ceph-mon[30677]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T08:09:37.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:37 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:37.830 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:37 smithi196 ceph-mon[30677]: from='osd.3 [v2:172.21.15.71:6826/2205658375,v1:172.21.15.71:6827/2205658375]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T08:09:37.830 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:37 smithi196 ceph-mon[30677]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T08:09:37.844 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:37 smithi071 ceph-mon[39897]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T08:09:37.845 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:37 smithi071 ceph-mon[39897]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T08:09:37.845 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:37 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:37.845 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:37 smithi071 ceph-mon[39897]: from='osd.3 [v2:172.21.15.71:6826/2205658375,v1:172.21.15.71:6827/2205658375]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T08:09:37.845 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:37 smithi071 ceph-mon[39897]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T08:09:37.845 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:37 smithi071 ceph-mon[25834]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T08:09:37.845 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:37 smithi071 ceph-mon[25834]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T08:09:37.846 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:37 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:37.846 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:37 smithi071 ceph-mon[25834]: from='osd.3 [v2:172.21.15.71:6826/2205658375,v1:172.21.15.71:6827/2205658375]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T08:09:37.846 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:37 smithi071 ceph-mon[25834]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T08:09:37.846 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:09:37 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-3[69590]: 2024-09-11T08:09:37.581+0000 7ff0cd1fb640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T08:09:37.846 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:09:37 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-3[69590]: 2024-09-11T08:09:37.584+0000 7ff0c602e640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:09:38.237 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/nvme1n1 2024-09-11T08:09:38.238 INFO:teuthology.orchestra.run.smithi196.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-11T08:09:38.241 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-11T08:09:38.256 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-09-11T08:09:38.257 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-09-11T08:09:38.257 INFO:teuthology.orchestra.run.smithi196.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128192 s, 818 MB/s 2024-09-11T08:09:38.257 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-09-11T08:09:38.823 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:38 smithi196 ceph-mon[30677]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:38.823 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:38 smithi196 ceph-mon[30677]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T08:09:38.824 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:38 smithi196 ceph-mon[30677]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T08:09:38.824 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:38 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:38.824 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:38 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:38.847 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi196:/dev/nvme1n1 2024-09-11T08:09:38.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:38 smithi071 ceph-mon[25834]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:38.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:38 smithi071 ceph-mon[25834]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T08:09:38.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:38 smithi071 ceph-mon[25834]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T08:09:38.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:38 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:38.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:38 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:38.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:38 smithi071 ceph-mon[39897]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:38.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:38 smithi071 ceph-mon[39897]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T08:09:38.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:38 smithi071 ceph-mon[39897]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T08:09:38.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:38 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:38.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:38 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:39.088 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:09:39.812 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:39 smithi196 ceph-mon[30677]: purged_snaps scrub starts 2024-09-11T08:09:39.812 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:39 smithi196 ceph-mon[30677]: purged_snaps scrub ok 2024-09-11T08:09:39.812 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:39 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:39.812 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:39 smithi196 ceph-mon[30677]: osd.3 [v2:172.21.15.71:6826/2205658375,v1:172.21.15.71:6827/2205658375] boot 2024-09-11T08:09:39.812 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:39 smithi196 ceph-mon[30677]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T08:09:39.812 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:39 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:39.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:39 smithi071 ceph-mon[39897]: purged_snaps scrub starts 2024-09-11T08:09:39.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:39 smithi071 ceph-mon[39897]: purged_snaps scrub ok 2024-09-11T08:09:39.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:39 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:39.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:39 smithi071 ceph-mon[39897]: osd.3 [v2:172.21.15.71:6826/2205658375,v1:172.21.15.71:6827/2205658375] boot 2024-09-11T08:09:39.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:39 smithi071 ceph-mon[39897]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T08:09:39.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:39 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:39.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:39 smithi071 ceph-mon[25834]: purged_snaps scrub starts 2024-09-11T08:09:39.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:39 smithi071 ceph-mon[25834]: purged_snaps scrub ok 2024-09-11T08:09:39.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:39 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:39.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:39 smithi071 ceph-mon[25834]: osd.3 [v2:172.21.15.71:6826/2205658375,v1:172.21.15.71:6827/2205658375] boot 2024-09-11T08:09:39.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:39 smithi071 ceph-mon[25834]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T08:09:39.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:39 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:09:40.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T08:09:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: Detected new or changed devices on smithi071 2024-09-11T08:09:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: Adjusting osd_memory_target on smithi071 to 4016M 2024-09-11T08:09:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:40.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:40.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:40.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:40.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:40.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:40 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:40.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:40.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T08:09:40.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: Detected new or changed devices on smithi071 2024-09-11T08:09:40.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:40.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: Adjusting osd_memory_target on smithi071 to 4016M 2024-09-11T08:09:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:40.947 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:09:40.947 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T08:09:40.947 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: Detected new or changed devices on smithi071 2024-09-11T08:09:40.947 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:40.947 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:40.947 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:40.947 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:40.947 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:40.947 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:09:40.947 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: Adjusting osd_memory_target on smithi071 to 4016M 2024-09-11T08:09:40.947 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:40.948 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:40.948 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:40.948 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:40.948 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:40.948 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:40.948 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:40.948 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:40 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:41.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:41 smithi196 ceph-mon[30677]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:09:41.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:41 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:09:41.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:41 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:09:41.830 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:41 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:41.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:41 smithi071 ceph-mon[25834]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:09:41.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:41 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:09:41.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:41 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:09:41.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:41 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:41.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:41 smithi071 ceph-mon[39897]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:09:41.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:41 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:09:41.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:41 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:09:41.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:41 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:42.752 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:42 smithi196 ceph-mon[30677]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:42.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:42 smithi071 ceph-mon[25834]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:42.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:42 smithi071 ceph-mon[39897]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:43.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:43 smithi196 ceph-mon[30677]: from='client.? 172.21.15.196:0/4284148470' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f35baaaa-6a0f-45ee-8946-1c4ad32bbd4e"}]: dispatch 2024-09-11T08:09:43.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:43 smithi196 ceph-mon[30677]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f35baaaa-6a0f-45ee-8946-1c4ad32bbd4e"}]: dispatch 2024-09-11T08:09:43.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:43 smithi196 ceph-mon[30677]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f35baaaa-6a0f-45ee-8946-1c4ad32bbd4e"}]': finished 2024-09-11T08:09:43.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:43 smithi196 ceph-mon[30677]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T08:09:43.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:43 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:09:43.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:43 smithi071 ceph-mon[25834]: from='client.? 172.21.15.196:0/4284148470' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f35baaaa-6a0f-45ee-8946-1c4ad32bbd4e"}]: dispatch 2024-09-11T08:09:43.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:43 smithi071 ceph-mon[25834]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f35baaaa-6a0f-45ee-8946-1c4ad32bbd4e"}]: dispatch 2024-09-11T08:09:43.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:43 smithi071 ceph-mon[25834]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f35baaaa-6a0f-45ee-8946-1c4ad32bbd4e"}]': finished 2024-09-11T08:09:43.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:43 smithi071 ceph-mon[25834]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T08:09:43.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:43 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:09:43.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:43 smithi071 ceph-mon[39897]: from='client.? 172.21.15.196:0/4284148470' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f35baaaa-6a0f-45ee-8946-1c4ad32bbd4e"}]: dispatch 2024-09-11T08:09:43.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:43 smithi071 ceph-mon[39897]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f35baaaa-6a0f-45ee-8946-1c4ad32bbd4e"}]: dispatch 2024-09-11T08:09:43.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:43 smithi071 ceph-mon[39897]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f35baaaa-6a0f-45ee-8946-1c4ad32bbd4e"}]': finished 2024-09-11T08:09:43.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:43 smithi071 ceph-mon[39897]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T08:09:43.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:43 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:09:45.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:45 smithi196 ceph-mon[30677]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:45.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:45 smithi196 ceph-mon[30677]: from='client.? 172.21.15.196:0/1890297624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:09:45.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:45 smithi071 ceph-mon[39897]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:45.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:45 smithi071 ceph-mon[39897]: from='client.? 172.21.15.196:0/1890297624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:09:45.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:45 smithi071 ceph-mon[25834]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:45.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:45 smithi071 ceph-mon[25834]: from='client.? 172.21.15.196:0/1890297624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:09:47.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:47 smithi196 ceph-mon[30677]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:47.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:47 smithi071 ceph-mon[39897]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:47.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:47 smithi071 ceph-mon[25834]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:49.427 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:49 smithi196 ceph-mon[30677]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:49.444 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:49 smithi071 ceph-mon[25834]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:49.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:49 smithi071 ceph-mon[39897]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:50.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:50 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T08:09:50.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:50 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:50.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:50 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T08:09:50.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:50 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:50.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:50 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T08:09:50.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:50 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:51.243 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:51 smithi196 ceph-mon[30677]: Deploying daemon osd.4 on smithi196 2024-09-11T08:09:51.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:51 smithi196 ceph-mon[30677]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:51.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:51 smithi071 ceph-mon[39897]: Deploying daemon osd.4 on smithi196 2024-09-11T08:09:51.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:51 smithi071 ceph-mon[39897]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:51.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:51 smithi071 ceph-mon[25834]: Deploying daemon osd.4 on smithi196 2024-09-11T08:09:51.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:51 smithi071 ceph-mon[25834]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:53.428 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:53 smithi196 ceph-mon[30677]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:53.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:53 smithi071 ceph-mon[39897]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:53.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:53 smithi071 ceph-mon[25834]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:55.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:55 smithi196 ceph-mon[30677]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:55.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:55 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:55.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:55 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:55.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:55 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:55.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:55 smithi071 ceph-mon[39897]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:55.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:55 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:55.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:55 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:55.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:55 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:55.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:55 smithi071 ceph-mon[25834]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:55.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:55 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:55.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:55 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:55.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:55 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:57.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:56 smithi196 ceph-mon[30677]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:57.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:57.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:57.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:57.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:57.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:57.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:56 smithi071 ceph-mon[25834]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:57.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:57.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:57.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:57.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:57.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:57.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:56 smithi071 ceph-mon[39897]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:57.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:57.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:57.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:57.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:57.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:57.549 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 4 on host 'smithi196' 2024-09-11T08:09:57.952 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:57 smithi071 ceph-mon[25834]: from='osd.4 [v2:172.21.15.196:6800/2383004810,v1:172.21.15.196:6801/2383004810]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:09:57.952 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:57 smithi071 ceph-mon[25834]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:09:57.952 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:57 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:57.953 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:57 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:57.953 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:57 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:57.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:57 smithi071 ceph-mon[39897]: from='osd.4 [v2:172.21.15.196:6800/2383004810,v1:172.21.15.196:6801/2383004810]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:09:57.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:57 smithi071 ceph-mon[39897]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:09:57.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:57 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:57.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:57 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:57.953 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:57 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:58.093 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:57 smithi196 ceph-mon[30677]: from='osd.4 [v2:172.21.15.196:6800/2383004810,v1:172.21.15.196:6801/2383004810]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:09:58.093 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:57 smithi196 ceph-mon[30677]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:09:58.093 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:57 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:58.093 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:57 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:58.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:57 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:09:58.114 DEBUG:teuthology.orchestra.run.smithi196:osd.4> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T08:09:58.117 INFO:tasks.cephadm:Deploying osd.5 on smithi196 with /dev/nvme2n1... 2024-09-11T08:09:58.117 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme2n1 2024-09-11T08:09:58.776 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:09:59.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:58 smithi196 ceph-mon[30677]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:59.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:58 smithi196 ceph-mon[30677]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T08:09:59.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:58 smithi196 ceph-mon[30677]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T08:09:59.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:58 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:09:59.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:58 smithi196 ceph-mon[30677]: from='osd.4 [v2:172.21.15.196:6800/2383004810,v1:172.21.15.196:6801/2383004810]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:09:59.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:58 smithi196 ceph-mon[30677]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:09:59.182 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:09:58 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4[36098]: 2024-09-11T08:09:58.859+0000 7fee60da9640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T08:09:59.182 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:09:58 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4[36098]: 2024-09-11T08:09:58.863+0000 7fee59bdc640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:09:59.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:58 smithi071 ceph-mon[25834]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:59.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:58 smithi071 ceph-mon[25834]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T08:09:59.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:58 smithi071 ceph-mon[25834]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T08:09:59.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:58 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:09:59.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:58 smithi071 ceph-mon[25834]: from='osd.4 [v2:172.21.15.196:6800/2383004810,v1:172.21.15.196:6801/2383004810]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:09:59.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:58 smithi071 ceph-mon[25834]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:09:59.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:58 smithi071 ceph-mon[39897]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:09:59.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:58 smithi071 ceph-mon[39897]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T08:09:59.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:58 smithi071 ceph-mon[39897]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T08:09:59.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:58 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:09:59.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:58 smithi071 ceph-mon[39897]: from='osd.4 [v2:172.21.15.196:6800/2383004810,v1:172.21.15.196:6801/2383004810]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:09:59.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:58 smithi071 ceph-mon[39897]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:10:00.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:59 smithi196 ceph-mon[30677]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T08:10:00.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:59 smithi196 ceph-mon[30677]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T08:10:00.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:59 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:10:00.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:59 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:10:00.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:59 smithi196 ceph-mon[30677]: osd.4 [v2:172.21.15.196:6800/2383004810,v1:172.21.15.196:6801/2383004810] boot 2024-09-11T08:10:00.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:59 smithi196 ceph-mon[30677]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T08:10:00.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:09:59 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:10:00.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:59 smithi071 ceph-mon[25834]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T08:10:00.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:59 smithi071 ceph-mon[25834]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T08:10:00.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:59 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:10:00.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:59 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:10:00.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:59 smithi071 ceph-mon[25834]: osd.4 [v2:172.21.15.196:6800/2383004810,v1:172.21.15.196:6801/2383004810] boot 2024-09-11T08:10:00.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:59 smithi071 ceph-mon[25834]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T08:10:00.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:09:59 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:10:00.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:59 smithi071 ceph-mon[39897]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T08:10:00.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:59 smithi071 ceph-mon[39897]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T08:10:00.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:59 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:10:00.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:59 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:10:00.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:59 smithi071 ceph-mon[39897]: osd.4 [v2:172.21.15.196:6800/2383004810,v1:172.21.15.196:6801/2383004810] boot 2024-09-11T08:10:00.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:59 smithi071 ceph-mon[39897]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T08:10:00.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:09:59 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:10:01.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:00 smithi196 ceph-mon[30677]: purged_snaps scrub starts 2024-09-11T08:10:01.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:00 smithi196 ceph-mon[30677]: purged_snaps scrub ok 2024-09-11T08:10:01.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:00 smithi196 ceph-mon[30677]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:10:01.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:00 smithi196 ceph-mon[30677]: overall HEALTH_OK 2024-09-11T08:10:01.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:00 smithi196 ceph-mon[30677]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T08:10:01.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:00 smithi071 ceph-mon[25834]: purged_snaps scrub starts 2024-09-11T08:10:01.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:00 smithi071 ceph-mon[25834]: purged_snaps scrub ok 2024-09-11T08:10:01.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:00 smithi071 ceph-mon[25834]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:10:01.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:00 smithi071 ceph-mon[25834]: overall HEALTH_OK 2024-09-11T08:10:01.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:00 smithi071 ceph-mon[25834]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T08:10:01.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:00 smithi071 ceph-mon[39897]: purged_snaps scrub starts 2024-09-11T08:10:01.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:00 smithi071 ceph-mon[39897]: purged_snaps scrub ok 2024-09-11T08:10:01.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:00 smithi071 ceph-mon[39897]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:10:01.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:00 smithi071 ceph-mon[39897]: overall HEALTH_OK 2024-09-11T08:10:01.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:00 smithi071 ceph-mon[39897]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T08:10:01.889 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/nvme2n1 2024-09-11T08:10:01.889 INFO:teuthology.orchestra.run.smithi196.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-11T08:10:01.893 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme2n1 bs=1M count=10 conv=fsync 2024-09-11T08:10:01.908 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-09-11T08:10:01.908 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-09-11T08:10:01.908 INFO:teuthology.orchestra.run.smithi196.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125624 s, 835 MB/s 2024-09-11T08:10:01.908 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-09-11T08:10:02.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:02 smithi196 ceph-mon[30677]: Detected new or changed devices on smithi196 2024-09-11T08:10:02.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:02 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:02.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:02 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:02.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:02 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:02.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:02 smithi196 ceph-mon[30677]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-11T08:10:02.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:02 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:02.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:02 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:02.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:02 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:02.484 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:02 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:02.484 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:02 smithi196 ceph-mon[30677]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:02.484 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:02 smithi196 ceph-mon[30677]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T08:10:02.507 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi196:/dev/nvme2n1 2024-09-11T08:10:02.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[25834]: Detected new or changed devices on smithi196 2024-09-11T08:10:02.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:02.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:02.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:02.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[25834]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-11T08:10:02.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:02.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:02.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:02.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:02.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[25834]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:02.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[25834]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T08:10:02.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[39897]: Detected new or changed devices on smithi196 2024-09-11T08:10:02.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:02.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:02.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:02.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[39897]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-11T08:10:02.697 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:02.697 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:02.697 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:02.697 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:02.697 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[39897]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:02.697 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:02 smithi071 ceph-mon[39897]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T08:10:02.743 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:10:03.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:03 smithi196 ceph-mon[30677]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T08:10:04.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:03 smithi071 ceph-mon[25834]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T08:10:04.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:03 smithi071 ceph-mon[39897]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T08:10:05.161 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:04 smithi196 ceph-mon[30677]: pgmap v115: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:05.161 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:04 smithi196 ceph-mon[30677]: osdmap e34: 5 total, 5 up, 5 in 2024-09-11T08:10:05.161 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:04 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:10:05.161 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:04 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:10:05.161 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:04 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:05.201 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:04 smithi071 ceph-mon[25834]: pgmap v115: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:05.201 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:04 smithi071 ceph-mon[25834]: osdmap e34: 5 total, 5 up, 5 in 2024-09-11T08:10:05.201 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:04 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:10:05.201 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:04 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:10:05.201 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:04 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:05.202 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:04 smithi071 ceph-mon[39897]: pgmap v115: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:05.202 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:04 smithi071 ceph-mon[39897]: osdmap e34: 5 total, 5 up, 5 in 2024-09-11T08:10:05.202 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:04 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:10:05.202 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:04 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:10:05.202 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:04 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:05.880 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:05 smithi196 ceph-mon[30677]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:10:06.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:05 smithi071 ceph-mon[25834]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:10:06.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:05 smithi071 ceph-mon[39897]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:10:07.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:06 smithi196 ceph-mon[30677]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-09-11T08:10:07.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:06 smithi196 ceph-mon[30677]: from='client.? 172.21.15.196:0/1132522781' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84bbc0da-e641-42f8-87eb-68da7f4dbd5e"}]: dispatch 2024-09-11T08:10:07.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:06 smithi196 ceph-mon[30677]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84bbc0da-e641-42f8-87eb-68da7f4dbd5e"}]: dispatch 2024-09-11T08:10:07.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:06 smithi196 ceph-mon[30677]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84bbc0da-e641-42f8-87eb-68da7f4dbd5e"}]': finished 2024-09-11T08:10:07.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:06 smithi196 ceph-mon[30677]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T08:10:07.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:06 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:10:07.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:06 smithi071 ceph-mon[39897]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-09-11T08:10:07.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:06 smithi071 ceph-mon[39897]: from='client.? 172.21.15.196:0/1132522781' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84bbc0da-e641-42f8-87eb-68da7f4dbd5e"}]: dispatch 2024-09-11T08:10:07.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:06 smithi071 ceph-mon[39897]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84bbc0da-e641-42f8-87eb-68da7f4dbd5e"}]: dispatch 2024-09-11T08:10:07.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:06 smithi071 ceph-mon[39897]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84bbc0da-e641-42f8-87eb-68da7f4dbd5e"}]': finished 2024-09-11T08:10:07.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:06 smithi071 ceph-mon[39897]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T08:10:07.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:06 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:10:07.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:06 smithi071 ceph-mon[25834]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-09-11T08:10:07.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:06 smithi071 ceph-mon[25834]: from='client.? 172.21.15.196:0/1132522781' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84bbc0da-e641-42f8-87eb-68da7f4dbd5e"}]: dispatch 2024-09-11T08:10:07.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:06 smithi071 ceph-mon[25834]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84bbc0da-e641-42f8-87eb-68da7f4dbd5e"}]: dispatch 2024-09-11T08:10:07.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:06 smithi071 ceph-mon[25834]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84bbc0da-e641-42f8-87eb-68da7f4dbd5e"}]': finished 2024-09-11T08:10:07.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:06 smithi071 ceph-mon[25834]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T08:10:07.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:06 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:10:09.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:08 smithi196 ceph-mon[30677]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-11T08:10:09.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:08 smithi196 ceph-mon[30677]: from='client.? 172.21.15.196:0/4291543144' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:10:09.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:08 smithi071 ceph-mon[25834]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-11T08:10:09.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:08 smithi071 ceph-mon[25834]: from='client.? 172.21.15.196:0/4291543144' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:10:09.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:08 smithi071 ceph-mon[39897]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-11T08:10:09.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:08 smithi071 ceph-mon[39897]: from='client.? 172.21.15.196:0/4291543144' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:10:11.176 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:10 smithi196 ceph-mon[30677]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-11T08:10:11.194 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:10 smithi071 ceph-mon[25834]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-11T08:10:11.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:10 smithi071 ceph-mon[39897]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-11T08:10:13.038 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:12 smithi196 ceph-mon[30677]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:10:13.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:12 smithi071 ceph-mon[25834]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:10:13.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:12 smithi071 ceph-mon[39897]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:10:14.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:13 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T08:10:14.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:13 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:14.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:13 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T08:10:14.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:13 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:14.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:13 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T08:10:14.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:13 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:15.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:14 smithi196 ceph-mon[30677]: Deploying daemon osd.5 on smithi196 2024-09-11T08:10:15.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:14 smithi196 ceph-mon[30677]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T08:10:15.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:14 smithi071 ceph-mon[25834]: Deploying daemon osd.5 on smithi196 2024-09-11T08:10:15.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:14 smithi071 ceph-mon[25834]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T08:10:15.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:14 smithi071 ceph-mon[39897]: Deploying daemon osd.5 on smithi196 2024-09-11T08:10:15.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:14 smithi071 ceph-mon[39897]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T08:10:17.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:16 smithi196 ceph-mon[30677]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:17.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:16 smithi071 ceph-mon[25834]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:17.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:16 smithi071 ceph-mon[39897]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:19.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:18 smithi196 ceph-mon[30677]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:19.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:18 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:19.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:18 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:19.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:18 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:19.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:18 smithi071 ceph-mon[25834]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:19.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:18 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:19.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:18 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:19.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:18 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:19.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:18 smithi071 ceph-mon[39897]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:19.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:18 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:19.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:18 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:19.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:18 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:21.092 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:20 smithi196 ceph-mon[30677]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:21.092 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:20 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:21.093 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:20 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:21.093 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:20 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:21.093 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:20 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:21.093 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:20 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:21.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:20 smithi071 ceph-mon[25834]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:21.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:20 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:21.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:20 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:21.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:20 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:21.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:20 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:21.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:20 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:21.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:20 smithi071 ceph-mon[39897]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:21.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:20 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:21.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:20 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:21.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:20 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:21.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:20 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:21.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:20 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:21.411 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 5 on host 'smithi196' 2024-09-11T08:10:22.012 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:21 smithi196 ceph-mon[30677]: from='osd.5 [v2:172.21.15.196:6808/3314011112,v1:172.21.15.196:6809/3314011112]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:10:22.012 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:21 smithi196 ceph-mon[30677]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:10:22.012 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:21 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:22.013 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:21 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:22.013 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:21 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:22.041 DEBUG:teuthology.orchestra.run.smithi196:osd.5> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T08:10:22.044 INFO:tasks.cephadm:Deploying osd.6 on smithi196 with /dev/nvme3n1... 2024-09-11T08:10:22.044 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme3n1 2024-09-11T08:10:22.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:21 smithi071 ceph-mon[25834]: from='osd.5 [v2:172.21.15.196:6808/3314011112,v1:172.21.15.196:6809/3314011112]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:10:22.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:21 smithi071 ceph-mon[25834]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:10:22.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:22.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:22.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:22.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:21 smithi071 ceph-mon[39897]: from='osd.5 [v2:172.21.15.196:6808/3314011112,v1:172.21.15.196:6809/3314011112]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:10:22.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:21 smithi071 ceph-mon[39897]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:10:22.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:21 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:22.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:21 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:22.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:21 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:23.018 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:22 smithi196 ceph-mon[30677]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:23.018 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:22 smithi196 ceph-mon[30677]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T08:10:23.018 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:22 smithi196 ceph-mon[30677]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T08:10:23.019 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:22 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:10:23.019 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:22 smithi196 ceph-mon[30677]: from='osd.5 [v2:172.21.15.196:6808/3314011112,v1:172.21.15.196:6809/3314011112]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:10:23.019 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:22 smithi196 ceph-mon[30677]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:10:23.019 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:10:22 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:10:22.909+0000 7fe563315640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T08:10:23.019 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:10:22 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:10:22.914+0000 7fe55a923640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:10:23.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:22 smithi071 ceph-mon[25834]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:23.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:22 smithi071 ceph-mon[25834]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T08:10:23.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:22 smithi071 ceph-mon[25834]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T08:10:23.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:22 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:10:23.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:22 smithi071 ceph-mon[25834]: from='osd.5 [v2:172.21.15.196:6808/3314011112,v1:172.21.15.196:6809/3314011112]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:10:23.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:22 smithi071 ceph-mon[25834]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:10:23.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:22 smithi071 ceph-mon[39897]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:23.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:22 smithi071 ceph-mon[39897]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T08:10:23.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:22 smithi071 ceph-mon[39897]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T08:10:23.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:22 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:10:23.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:22 smithi071 ceph-mon[39897]: from='osd.5 [v2:172.21.15.196:6808/3314011112,v1:172.21.15.196:6809/3314011112]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:10:23.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:22 smithi071 ceph-mon[39897]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:10:23.468 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:10:24.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:23 smithi196 ceph-mon[30677]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T08:10:24.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:23 smithi196 ceph-mon[30677]: osdmap e37: 6 total, 5 up, 6 in 2024-09-11T08:10:24.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:23 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:10:24.184 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:23 smithi196 ceph-mon[30677]: osd.5 [v2:172.21.15.196:6808/3314011112,v1:172.21.15.196:6809/3314011112] boot 2024-09-11T08:10:24.185 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:23 smithi196 ceph-mon[30677]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T08:10:24.185 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:23 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:10:24.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:23 smithi071 ceph-mon[25834]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T08:10:24.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:23 smithi071 ceph-mon[25834]: osdmap e37: 6 total, 5 up, 6 in 2024-09-11T08:10:24.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:23 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:10:24.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:23 smithi071 ceph-mon[25834]: osd.5 [v2:172.21.15.196:6808/3314011112,v1:172.21.15.196:6809/3314011112] boot 2024-09-11T08:10:24.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:23 smithi071 ceph-mon[25834]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T08:10:24.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:23 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:10:24.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:23 smithi071 ceph-mon[39897]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T08:10:24.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:23 smithi071 ceph-mon[39897]: osdmap e37: 6 total, 5 up, 6 in 2024-09-11T08:10:24.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:23 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:10:24.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:23 smithi071 ceph-mon[39897]: osd.5 [v2:172.21.15.196:6808/3314011112,v1:172.21.15.196:6809/3314011112] boot 2024-09-11T08:10:24.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:23 smithi071 ceph-mon[39897]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T08:10:24.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:23 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:10:25.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:24 smithi196 ceph-mon[30677]: purged_snaps scrub starts 2024-09-11T08:10:25.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:24 smithi196 ceph-mon[30677]: purged_snaps scrub ok 2024-09-11T08:10:25.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:24 smithi196 ceph-mon[30677]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:25.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:24 smithi196 ceph-mon[30677]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T08:10:25.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:24 smithi071 ceph-mon[25834]: purged_snaps scrub starts 2024-09-11T08:10:25.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:24 smithi071 ceph-mon[25834]: purged_snaps scrub ok 2024-09-11T08:10:25.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:24 smithi071 ceph-mon[25834]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:25.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:24 smithi071 ceph-mon[25834]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T08:10:25.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:24 smithi071 ceph-mon[39897]: purged_snaps scrub starts 2024-09-11T08:10:25.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:24 smithi071 ceph-mon[39897]: purged_snaps scrub ok 2024-09-11T08:10:25.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:24 smithi071 ceph-mon[39897]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:10:25.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:24 smithi071 ceph-mon[39897]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T08:10:25.910 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/nvme3n1 2024-09-11T08:10:25.911 INFO:teuthology.orchestra.run.smithi196.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-11T08:10:25.916 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme3n1 bs=1M count=10 conv=fsync 2024-09-11T08:10:25.931 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-09-11T08:10:25.931 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-09-11T08:10:25.931 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0139609 s, 751 MB/s 2024-09-11T08:10:25.931 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-09-11T08:10:26.530 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:26 smithi196 ceph-mon[30677]: Detected new or changed devices on smithi196 2024-09-11T08:10:26.530 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:26 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:26.530 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:26 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:26.530 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:26 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:26.530 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:26 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:26.530 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:26 smithi196 ceph-mon[30677]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-11T08:10:26.530 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:26 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:26.531 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:26 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:26.531 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:26 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:26.531 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:26 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:26.531 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:26 smithi196 ceph-mon[30677]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:26.531 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:26 smithi196 ceph-mon[30677]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T08:10:26.553 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi196:/dev/nvme3n1 2024-09-11T08:10:26.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[39897]: Detected new or changed devices on smithi196 2024-09-11T08:10:26.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:26.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:26.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:26.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:26.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[39897]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-11T08:10:26.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:26.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:26.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:26.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:26.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[39897]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:26.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[39897]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T08:10:26.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[25834]: Detected new or changed devices on smithi196 2024-09-11T08:10:26.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:26.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:26.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:26.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:26.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[25834]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-11T08:10:26.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:26.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:26.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:26.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:26.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[25834]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:26.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:26 smithi071 ceph-mon[25834]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T08:10:26.791 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:10:27.996 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:27 smithi196 ceph-mon[30677]: osdmap e41: 6 total, 6 up, 6 in 2024-09-11T08:10:28.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:27 smithi071 ceph-mon[25834]: osdmap e41: 6 total, 6 up, 6 in 2024-09-11T08:10:28.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:27 smithi071 ceph-mon[39897]: osdmap e41: 6 total, 6 up, 6 in 2024-09-11T08:10:29.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:28 smithi196 ceph-mon[30677]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:29.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:28 smithi196 ceph-mon[30677]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:10:29.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:28 smithi196 ceph-mon[30677]: osdmap e42: 6 total, 6 up, 6 in 2024-09-11T08:10:29.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:28 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:10:29.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:28 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:10:29.033 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:28 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:29.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:28 smithi071 ceph-mon[39897]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:29.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:28 smithi071 ceph-mon[39897]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:10:29.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:28 smithi071 ceph-mon[39897]: osdmap e42: 6 total, 6 up, 6 in 2024-09-11T08:10:29.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:28 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:10:29.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:28 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:10:29.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:28 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:29.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:28 smithi071 ceph-mon[25834]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:29.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:28 smithi071 ceph-mon[25834]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:10:29.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:28 smithi071 ceph-mon[25834]: osdmap e42: 6 total, 6 up, 6 in 2024-09-11T08:10:29.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:28 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:10:29.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:28 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:10:29.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:28 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:29.944 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:29 smithi071 ceph-mon[39897]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:10:29.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:29 smithi071 ceph-mon[25834]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:10:30.045 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:29 smithi196 ceph-mon[30677]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:10:31.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:30 smithi196 ceph-mon[30677]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:31.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:30 smithi196 ceph-mon[30677]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:10:31.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:30 smithi196 ceph-mon[30677]: Cluster is now healthy 2024-09-11T08:10:31.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:30 smithi196 ceph-mon[30677]: from='client.? 172.21.15.196:0/1397435094' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cff32ed5-909b-4fac-9d6d-1ea4ef27d4b2"}]: dispatch 2024-09-11T08:10:31.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:30 smithi196 ceph-mon[30677]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cff32ed5-909b-4fac-9d6d-1ea4ef27d4b2"}]: dispatch 2024-09-11T08:10:31.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:30 smithi196 ceph-mon[30677]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cff32ed5-909b-4fac-9d6d-1ea4ef27d4b2"}]': finished 2024-09-11T08:10:31.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:30 smithi196 ceph-mon[30677]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T08:10:31.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:30 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:31.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:30 smithi071 ceph-mon[25834]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:31.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:30 smithi071 ceph-mon[25834]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:10:31.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:30 smithi071 ceph-mon[25834]: Cluster is now healthy 2024-09-11T08:10:31.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:30 smithi071 ceph-mon[25834]: from='client.? 172.21.15.196:0/1397435094' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cff32ed5-909b-4fac-9d6d-1ea4ef27d4b2"}]: dispatch 2024-09-11T08:10:31.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:30 smithi071 ceph-mon[25834]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cff32ed5-909b-4fac-9d6d-1ea4ef27d4b2"}]: dispatch 2024-09-11T08:10:31.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:30 smithi071 ceph-mon[25834]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cff32ed5-909b-4fac-9d6d-1ea4ef27d4b2"}]': finished 2024-09-11T08:10:31.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:30 smithi071 ceph-mon[25834]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T08:10:31.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:30 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:31.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:30 smithi071 ceph-mon[39897]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:31.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:30 smithi071 ceph-mon[39897]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:10:31.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:30 smithi071 ceph-mon[39897]: Cluster is now healthy 2024-09-11T08:10:31.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:30 smithi071 ceph-mon[39897]: from='client.? 172.21.15.196:0/1397435094' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cff32ed5-909b-4fac-9d6d-1ea4ef27d4b2"}]: dispatch 2024-09-11T08:10:31.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:30 smithi071 ceph-mon[39897]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cff32ed5-909b-4fac-9d6d-1ea4ef27d4b2"}]: dispatch 2024-09-11T08:10:31.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:30 smithi071 ceph-mon[39897]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cff32ed5-909b-4fac-9d6d-1ea4ef27d4b2"}]': finished 2024-09-11T08:10:31.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:30 smithi071 ceph-mon[39897]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T08:10:31.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:30 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:33.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:32 smithi196 ceph-mon[30677]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-11T08:10:33.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:32 smithi196 ceph-mon[30677]: from='client.? 172.21.15.196:0/1341162508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:10:33.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:32 smithi071 ceph-mon[25834]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-11T08:10:33.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:32 smithi071 ceph-mon[25834]: from='client.? 172.21.15.196:0/1341162508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:10:33.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:32 smithi071 ceph-mon[39897]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-11T08:10:33.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:32 smithi071 ceph-mon[39897]: from='client.? 172.21.15.196:0/1341162508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:10:35.116 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:34 smithi196 ceph-mon[30677]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-11T08:10:35.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:34 smithi071 ceph-mon[25834]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-11T08:10:35.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:34 smithi071 ceph-mon[39897]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-11T08:10:36.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:36 smithi196 ceph-mon[30677]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:10:37.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:36 smithi071 ceph-mon[25834]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:10:37.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:36 smithi071 ceph-mon[39897]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:10:37.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:37 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T08:10:37.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:37 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:38.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:37 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T08:10:38.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:37 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:38.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:37 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T08:10:38.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:37 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:38.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:38 smithi196 ceph-mon[30677]: Deploying daemon osd.6 on smithi196 2024-09-11T08:10:38.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:38 smithi196 ceph-mon[30677]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T08:10:39.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:38 smithi071 ceph-mon[25834]: Deploying daemon osd.6 on smithi196 2024-09-11T08:10:39.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:38 smithi071 ceph-mon[25834]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T08:10:39.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:38 smithi071 ceph-mon[39897]: Deploying daemon osd.6 on smithi196 2024-09-11T08:10:39.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:38 smithi071 ceph-mon[39897]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T08:10:41.120 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:40 smithi196 ceph-mon[30677]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:41.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:40 smithi071 ceph-mon[25834]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:41.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:40 smithi071 ceph-mon[39897]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:42.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:42 smithi196 ceph-mon[30677]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:42.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:42 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:42.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:42 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:42.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:42 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:43.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:42 smithi071 ceph-mon[25834]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:43.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:42 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:43.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:42 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:43.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:42 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:43.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:42 smithi071 ceph-mon[39897]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:43.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:43.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:43.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:44.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:44 smithi196 ceph-mon[30677]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:45.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:44 smithi071 ceph-mon[25834]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:45.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:44 smithi071 ceph-mon[39897]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:45.277 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 6 on host 'smithi196' 2024-09-11T08:10:45.851 DEBUG:teuthology.orchestra.run.smithi196:osd.6> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T08:10:45.854 INFO:tasks.cephadm:Deploying osd.7 on smithi196 with /dev/nvme4n1... 2024-09-11T08:10:45.855 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme4n1 2024-09-11T08:10:46.150 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:10:46.631 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:46 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:46.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:46 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:46.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:46 smithi196 ceph-mon[30677]: from='osd.6 [v2:172.21.15.196:6816/4073919265,v1:172.21.15.196:6817/4073919265]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:10:46.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:46 smithi196 ceph-mon[30677]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:10:46.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:46 smithi196 ceph-mon[30677]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:46.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:46 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:46.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:46 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:46.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:46 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:46.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:46 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:46.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:46 smithi071 ceph-mon[39897]: from='osd.6 [v2:172.21.15.196:6816/4073919265,v1:172.21.15.196:6817/4073919265]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:10:46.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:46 smithi071 ceph-mon[39897]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:10:46.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:46 smithi071 ceph-mon[39897]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:46.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:46 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:46.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:46 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:46.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:46 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:46.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:46 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:46.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:46 smithi071 ceph-mon[25834]: from='osd.6 [v2:172.21.15.196:6816/4073919265,v1:172.21.15.196:6817/4073919265]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:10:46.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:46 smithi071 ceph-mon[25834]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:10:46.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:46 smithi071 ceph-mon[25834]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:46.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:46 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:46.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:46 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:47.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:47 smithi196 ceph-mon[30677]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T08:10:47.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:47 smithi196 ceph-mon[30677]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T08:10:47.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:47 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:47.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:47 smithi196 ceph-mon[30677]: from='osd.6 [v2:172.21.15.196:6816/4073919265,v1:172.21.15.196:6817/4073919265]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:10:47.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:47 smithi196 ceph-mon[30677]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:10:47.589 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:10:47 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:10:47.292+0000 7efca33cd640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T08:10:47.589 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:10:47 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:10:47.300+0000 7efc9e204640 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:10:47.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:47 smithi071 ceph-mon[25834]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T08:10:47.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:47 smithi071 ceph-mon[25834]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T08:10:47.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:47 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:47.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:47 smithi071 ceph-mon[25834]: from='osd.6 [v2:172.21.15.196:6816/4073919265,v1:172.21.15.196:6817/4073919265]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:10:47.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:47 smithi071 ceph-mon[25834]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:10:47.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:47 smithi071 ceph-mon[39897]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T08:10:47.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:47 smithi071 ceph-mon[39897]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T08:10:47.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:47 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:47.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:47 smithi071 ceph-mon[39897]: from='osd.6 [v2:172.21.15.196:6816/4073919265,v1:172.21.15.196:6817/4073919265]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:10:47.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:47 smithi071 ceph-mon[39897]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:10:47.911 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/nvme4n1 2024-09-11T08:10:47.911 INFO:teuthology.orchestra.run.smithi196.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-11T08:10:47.916 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-09-11T08:10:47.932 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-09-11T08:10:47.932 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-09-11T08:10:47.932 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0137617 s, 762 MB/s 2024-09-11T08:10:47.932 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-09-11T08:10:48.553 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:48 smithi196 ceph-mon[30677]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T08:10:48.553 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:48 smithi196 ceph-mon[30677]: osdmap e45: 7 total, 6 up, 7 in 2024-09-11T08:10:48.553 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:48 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:48.553 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:48 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:48.553 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:48 smithi196 ceph-mon[30677]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:48.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:48 smithi071 ceph-mon[25834]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T08:10:48.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:48 smithi071 ceph-mon[25834]: osdmap e45: 7 total, 6 up, 7 in 2024-09-11T08:10:48.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:48 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:48.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:48 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:48.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:48 smithi071 ceph-mon[25834]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:48.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:48 smithi071 ceph-mon[39897]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T08:10:48.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:48 smithi071 ceph-mon[39897]: osdmap e45: 7 total, 6 up, 7 in 2024-09-11T08:10:48.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:48 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:48.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:48 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:48.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:48 smithi071 ceph-mon[39897]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:10:49.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:49 smithi196 ceph-mon[30677]: purged_snaps scrub starts 2024-09-11T08:10:49.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:49 smithi196 ceph-mon[30677]: purged_snaps scrub ok 2024-09-11T08:10:49.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:49 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:49.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:49 smithi196 ceph-mon[30677]: osd.6 [v2:172.21.15.196:6816/4073919265,v1:172.21.15.196:6817/4073919265] boot 2024-09-11T08:10:49.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:49 smithi196 ceph-mon[30677]: osdmap e46: 7 total, 7 up, 7 in 2024-09-11T08:10:49.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:49 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:49.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:49 smithi196 ceph-mon[30677]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T08:10:49.456 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi196:/dev/nvme4n1 2024-09-11T08:10:49.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:49 smithi071 ceph-mon[25834]: purged_snaps scrub starts 2024-09-11T08:10:49.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:49 smithi071 ceph-mon[25834]: purged_snaps scrub ok 2024-09-11T08:10:49.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:49 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:49.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:49 smithi071 ceph-mon[25834]: osd.6 [v2:172.21.15.196:6816/4073919265,v1:172.21.15.196:6817/4073919265] boot 2024-09-11T08:10:49.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:49 smithi071 ceph-mon[25834]: osdmap e46: 7 total, 7 up, 7 in 2024-09-11T08:10:49.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:49 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:49.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:49 smithi071 ceph-mon[25834]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T08:10:49.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:49 smithi071 ceph-mon[39897]: purged_snaps scrub starts 2024-09-11T08:10:49.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:49 smithi071 ceph-mon[39897]: purged_snaps scrub ok 2024-09-11T08:10:49.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:49 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:49.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:49 smithi071 ceph-mon[39897]: osd.6 [v2:172.21.15.196:6816/4073919265,v1:172.21.15.196:6817/4073919265] boot 2024-09-11T08:10:49.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:49 smithi071 ceph-mon[39897]: osdmap e46: 7 total, 7 up, 7 in 2024-09-11T08:10:49.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:49 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:10:49.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:49 smithi071 ceph-mon[39897]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T08:10:49.720 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:10:51.133 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:50 smithi196 ceph-mon[30677]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:10:51.133 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:50 smithi196 ceph-mon[30677]: osdmap e48: 7 total, 7 up, 7 in 2024-09-11T08:10:51.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:50 smithi071 ceph-mon[25834]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:10:51.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:50 smithi071 ceph-mon[25834]: osdmap e48: 7 total, 7 up, 7 in 2024-09-11T08:10:51.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:50 smithi071 ceph-mon[39897]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:10:51.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:50 smithi071 ceph-mon[39897]: osdmap e48: 7 total, 7 up, 7 in 2024-09-11T08:10:52.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: Detected new or changed devices on smithi196 2024-09-11T08:10:52.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:52.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:52.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:52.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:52.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:52.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-11T08:10:52.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:52.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:52.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:52.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:52.433 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:52.433 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:52.433 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:52.433 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:52.433 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: osdmap e49: 7 total, 7 up, 7 in 2024-09-11T08:10:52.433 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:52 smithi196 ceph-mon[30677]: pgmap v155: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T08:10:52.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: Detected new or changed devices on smithi196 2024-09-11T08:10:52.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:52.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:52.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:52.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:52.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:52.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-11T08:10:52.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:52.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:52.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:52.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:52.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:52.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:52.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:52.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:52.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: osdmap e49: 7 total, 7 up, 7 in 2024-09-11T08:10:52.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[25834]: pgmap v155: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T08:10:52.447 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: Detected new or changed devices on smithi196 2024-09-11T08:10:52.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:52.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:52.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:52.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:52.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:10:52.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-11T08:10:52.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:52.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:52.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:52.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:52.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:52.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:52.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:52.449 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:10:52.449 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: osdmap e49: 7 total, 7 up, 7 in 2024-09-11T08:10:52.449 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:52 smithi071 ceph-mon[39897]: pgmap v155: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T08:10:53.646 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:53 smithi196 ceph-mon[30677]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:10:53.646 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:53 smithi196 ceph-mon[30677]: osdmap e50: 7 total, 7 up, 7 in 2024-09-11T08:10:53.646 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:53 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:10:53.646 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:53 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:10:53.647 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:53 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:53.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:53 smithi071 ceph-mon[25834]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:10:53.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:53 smithi071 ceph-mon[25834]: osdmap e50: 7 total, 7 up, 7 in 2024-09-11T08:10:53.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:53 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:10:53.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:53 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:10:53.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:53 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:53.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:53 smithi071 ceph-mon[39897]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:10:53.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:53 smithi071 ceph-mon[39897]: osdmap e50: 7 total, 7 up, 7 in 2024-09-11T08:10:53.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:53 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:10:53.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:53 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:10:53.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:53 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:54.648 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:54 smithi196 ceph-mon[30677]: pgmap v157: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-09-11T08:10:54.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:54 smithi071 ceph-mon[25834]: pgmap v157: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-09-11T08:10:54.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:54 smithi071 ceph-mon[39897]: pgmap v157: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-09-11T08:10:55.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:55 smithi196 ceph-mon[30677]: from='client.? 172.21.15.196:0/1269865423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1871ecef-9cb6-4705-9c26-032088cf97c1"}]: dispatch 2024-09-11T08:10:55.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:55 smithi196 ceph-mon[30677]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1871ecef-9cb6-4705-9c26-032088cf97c1"}]: dispatch 2024-09-11T08:10:55.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:55 smithi196 ceph-mon[30677]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1871ecef-9cb6-4705-9c26-032088cf97c1"}]': finished 2024-09-11T08:10:55.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:55 smithi196 ceph-mon[30677]: osdmap e51: 8 total, 7 up, 8 in 2024-09-11T08:10:55.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:55 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:10:55.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:55 smithi071 ceph-mon[25834]: from='client.? 172.21.15.196:0/1269865423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1871ecef-9cb6-4705-9c26-032088cf97c1"}]: dispatch 2024-09-11T08:10:55.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:55 smithi071 ceph-mon[25834]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1871ecef-9cb6-4705-9c26-032088cf97c1"}]: dispatch 2024-09-11T08:10:55.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:55 smithi071 ceph-mon[25834]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1871ecef-9cb6-4705-9c26-032088cf97c1"}]': finished 2024-09-11T08:10:55.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:55 smithi071 ceph-mon[25834]: osdmap e51: 8 total, 7 up, 8 in 2024-09-11T08:10:55.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:55 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:10:55.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:55 smithi071 ceph-mon[39897]: from='client.? 172.21.15.196:0/1269865423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1871ecef-9cb6-4705-9c26-032088cf97c1"}]: dispatch 2024-09-11T08:10:55.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:55 smithi071 ceph-mon[39897]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1871ecef-9cb6-4705-9c26-032088cf97c1"}]: dispatch 2024-09-11T08:10:55.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:55 smithi071 ceph-mon[39897]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1871ecef-9cb6-4705-9c26-032088cf97c1"}]': finished 2024-09-11T08:10:55.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:55 smithi071 ceph-mon[39897]: osdmap e51: 8 total, 7 up, 8 in 2024-09-11T08:10:55.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:55 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:10:56.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:56 smithi196 ceph-mon[30677]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-11T08:10:56.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:56 smithi196 ceph-mon[30677]: from='client.? 172.21.15.196:0/889894513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:10:56.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:56 smithi071 ceph-mon[25834]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-11T08:10:56.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:56 smithi071 ceph-mon[25834]: from='client.? 172.21.15.196:0/889894513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:10:56.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:56 smithi071 ceph-mon[39897]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-11T08:10:56.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:56 smithi071 ceph-mon[39897]: from='client.? 172.21.15.196:0/889894513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:10:58.867 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:10:58 smithi196 ceph-mon[30677]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:10:58.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:10:58 smithi071 ceph-mon[25834]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:10:58.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:10:58 smithi071 ceph-mon[39897]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:00.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:00 smithi196 ceph-mon[30677]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:00.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:00 smithi071 ceph-mon[25834]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:00.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:00 smithi071 ceph-mon[39897]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:01.855 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:01 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:11:01.855 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:01 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:01.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:01 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:11:01.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:01 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:01.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:01 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:11:01.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:01 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:02.840 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:02 smithi196 ceph-mon[30677]: Deploying daemon osd.7 on smithi196 2024-09-11T08:11:02.840 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:02 smithi196 ceph-mon[30677]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:02.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:02 smithi071 ceph-mon[25834]: Deploying daemon osd.7 on smithi196 2024-09-11T08:11:02.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:02 smithi071 ceph-mon[25834]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:02.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:02 smithi071 ceph-mon[39897]: Deploying daemon osd.7 on smithi196 2024-09-11T08:11:02.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:02 smithi071 ceph-mon[39897]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:04.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:04 smithi196 ceph-mon[30677]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:04.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:04 smithi071 ceph-mon[25834]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:04.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:04 smithi071 ceph-mon[39897]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:06.839 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:06 smithi196 ceph-mon[30677]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:06.839 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:06 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:06.839 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:06 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:06.839 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:06 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:06.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:06 smithi071 ceph-mon[25834]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:06.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:06 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:06.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:06 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:06.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:06 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:06.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:06 smithi071 ceph-mon[39897]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:06.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:06 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:06.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:06 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:06.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:06 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:08.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:08 smithi196 ceph-mon[30677]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:08.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:08 smithi071 ceph-mon[25834]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:08.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:08 smithi071 ceph-mon[39897]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:09.711 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 7 on host 'smithi196' 2024-09-11T08:11:10.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:09 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:10.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:09 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:10.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:09 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:10.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:09 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:10.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:09 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:10.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:09 smithi196 ceph-mon[30677]: from='osd.7 [v2:172.21.15.196:6824/3168217828,v1:172.21.15.196:6825/3168217828]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:11:10.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:09 smithi196 ceph-mon[30677]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:11:10.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:09 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:10.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:09 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:10.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:09 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:10.383 DEBUG:teuthology.orchestra.run.smithi196:osd.7> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T08:11:10.386 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-11T08:11:10.386 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T08:11:10.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:10.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:10.415 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:10.415 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:10.415 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:10.415 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[25834]: from='osd.7 [v2:172.21.15.196:6824/3168217828,v1:172.21.15.196:6825/3168217828]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:11:10.415 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[25834]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:11:10.415 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:10.415 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:10.415 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:10.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:10.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:10.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:10.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:10.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:10.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[39897]: from='osd.7 [v2:172.21.15.196:6824/3168217828,v1:172.21.15.196:6825/3168217828]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:11:10.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[39897]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:11:10.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:10.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:10.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:09 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:10.638 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:11:11.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:11 smithi196 ceph-mon[30677]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:11.375 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:11 smithi196 ceph-mon[30677]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:11:11.375 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:11 smithi196 ceph-mon[30677]: osdmap e52: 8 total, 7 up, 8 in 2024-09-11T08:11:11.375 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:11 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:11:11.375 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:11 smithi196 ceph-mon[30677]: from='osd.7 [v2:172.21.15.196:6824/3168217828,v1:172.21.15.196:6825/3168217828]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:11:11.375 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:11 smithi196 ceph-mon[30677]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:11:11.375 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:11:11 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:11:11.017+0000 7fea23d09640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T08:11:11.375 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:11:11 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:11:11.023+0000 7fea1db1c640 -1 osd.7 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:11:11.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:11 smithi071 ceph-mon[39897]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:11.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:11 smithi071 ceph-mon[39897]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:11:11.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:11 smithi071 ceph-mon[39897]: osdmap e52: 8 total, 7 up, 8 in 2024-09-11T08:11:11.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:11 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:11:11.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:11 smithi071 ceph-mon[39897]: from='osd.7 [v2:172.21.15.196:6824/3168217828,v1:172.21.15.196:6825/3168217828]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:11:11.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:11 smithi071 ceph-mon[39897]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:11:11.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:11 smithi071 ceph-mon[25834]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:11.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:11 smithi071 ceph-mon[25834]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:11:11.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:11 smithi071 ceph-mon[25834]: osdmap e52: 8 total, 7 up, 8 in 2024-09-11T08:11:11.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:11 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:11:11.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:11 smithi071 ceph-mon[25834]: from='osd.7 [v2:172.21.15.196:6824/3168217828,v1:172.21.15.196:6825/3168217828]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:11:11.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:11 smithi071 ceph-mon[25834]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T08:11:12.132 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:12 smithi071 ceph-mon[39897]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T08:11:12.132 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:12 smithi071 ceph-mon[39897]: osdmap e53: 8 total, 7 up, 8 in 2024-09-11T08:11:12.132 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:12 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:11:12.132 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:12 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:11:12.133 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:12 smithi071 ceph-mon[25834]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T08:11:12.133 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:12 smithi071 ceph-mon[25834]: osdmap e53: 8 total, 7 up, 8 in 2024-09-11T08:11:12.133 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:12 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:11:12.133 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:12 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:11:12.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:12 smithi196 ceph-mon[30677]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T08:11:12.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:12 smithi196 ceph-mon[30677]: osdmap e53: 8 total, 7 up, 8 in 2024-09-11T08:11:12.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:12 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:11:12.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:12 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:11:12.487 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:11:13.167 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:13 smithi071 ceph-mon[39897]: purged_snaps scrub starts 2024-09-11T08:11:13.167 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:13 smithi071 ceph-mon[39897]: purged_snaps scrub ok 2024-09-11T08:11:13.167 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:13 smithi071 ceph-mon[39897]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:13.167 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:13 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:11:13.167 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:13 smithi071 ceph-mon[39897]: osd.7 [v2:172.21.15.196:6824/3168217828,v1:172.21.15.196:6825/3168217828] boot 2024-09-11T08:11:13.167 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:13 smithi071 ceph-mon[39897]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:11:13.167 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:13 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:11:13.167 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:13 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/2745176972' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:11:13.168 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:13 smithi071 ceph-mon[25834]: purged_snaps scrub starts 2024-09-11T08:11:13.168 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:13 smithi071 ceph-mon[25834]: purged_snaps scrub ok 2024-09-11T08:11:13.168 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:13 smithi071 ceph-mon[25834]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:13.168 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:13 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:11:13.168 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:13 smithi071 ceph-mon[25834]: osd.7 [v2:172.21.15.196:6824/3168217828,v1:172.21.15.196:6825/3168217828] boot 2024-09-11T08:11:13.168 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:13 smithi071 ceph-mon[25834]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:11:13.168 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:13 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:11:13.168 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:13 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2745176972' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:11:13.192 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":54,"num_osds":8,"num_up_osds":8,"osd_up_since":1726042272,"num_in_osds":8,"osd_in_since":1726042254,"num_remapped_pgs":1} 2024-09-11T08:11:13.192 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:11:13.373 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:13 smithi196 ceph-mon[30677]: purged_snaps scrub starts 2024-09-11T08:11:13.373 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:13 smithi196 ceph-mon[30677]: purged_snaps scrub ok 2024-09-11T08:11:13.373 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:13 smithi196 ceph-mon[30677]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:11:13.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:13 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:11:13.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:13 smithi196 ceph-mon[30677]: osd.7 [v2:172.21.15.196:6824/3168217828,v1:172.21.15.196:6825/3168217828] boot 2024-09-11T08:11:13.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:13 smithi196 ceph-mon[30677]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:11:13.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:13 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:11:13.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:13 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/2745176972' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:11:13.485 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:11:14.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:14 smithi196 ceph-mon[30677]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:11:14.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:14 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:14.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:14 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:14.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:14 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:14.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:14 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:14.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:14 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:14.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:14 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:14.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:14 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:14.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:14 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:14.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:14 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:14.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:14 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:14.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[39897]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:11:14.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:14.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:14.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:14.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:14.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:14.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:14.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:14.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:14.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:14.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:14.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[25834]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:11:14.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:14.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:14.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:14.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:14.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:14.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:14.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:14.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:14.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:14.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:14 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:15.335 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:11:15.335 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:15 smithi071 ceph-mon[39897]: Detected new or changed devices on smithi196 2024-09-11T08:11:15.335 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:15 smithi071 ceph-mon[39897]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-11T08:11:15.335 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:15 smithi071 ceph-mon[39897]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:15.335 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:15 smithi071 ceph-mon[39897]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:11:15.336 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":57,"fsid":"566367e6-7014-11ef-bcea-c7b262605968","created":"2024-09-11T08:06:08.948724+0000","modified":"2024-09-11T08:11:15.031807+0000","last_up_change":"2024-09-11T08:11:12.015215+0000","last_in_change":"2024-09-11T08:10:54.456455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:09:13.770764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"b831aea3-cb55-49cc-ad5c-3c340ec2f249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1906336816},{"type":"v1","addr":"172.21.15.71:6803","nonce":1906336816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1906336816},{"type":"v1","addr":"172.21.15.71:6805","nonce":1906336816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1906336816},{"type":"v1","addr":"172.21.15.71:6809","nonce":1906336816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1906336816},{"type":"v1","addr":"172.21.15.71:6807","nonce":1906336816}]},"public_addr":"172.21.15.71:6803/1906336816","cluster_addr":"172.21.15.71:6805/1906336816","heartbeat_back_addr":"172.21.15.71:6809/1906336816","heartbeat_front_addr":"172.21.15.71:6807/1906336816","state":["exists","up"]},{"osd":1,"uuid":"168eac60-ee21-42ca-85ea-844e9073441d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1800678301},{"type":"v1","addr":"172.21.15.71:6811","nonce":1800678301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1800678301},{"type":"v1","addr":"172.21.15.71:6813","nonce":1800678301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1800678301},{"type":"v1","addr":"172.21.15.71:6817","nonce":1800678301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1800678301},{"type":"v1","addr":"172.21.15.71:6815","nonce":1800678301}]},"public_addr":"172.21.15.71:6811/1800678301","cluster_addr":"172.21.15.71:6813/1800678301","heartbeat_back_addr":"172.21.15.71:6817/1800678301","heartbeat_front_addr":"172.21.15.71:6815/1800678301","state":["exists","up"]},{"osd":2,"uuid":"4c3bbd0e-1a4a-42ad-8cd8-29c663141966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1556215465},{"type":"v1","addr":"172.21.15.71:6819","nonce":1556215465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1556215465},{"type":"v1","addr":"172.21.15.71:6821","nonce":1556215465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1556215465},{"type":"v1","addr":"172.21.15.71:6825","nonce":1556215465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1556215465},{"type":"v1","addr":"172.21.15.71:6823","nonce":1556215465}]},"public_addr":"172.21.15.71:6819/1556215465","cluster_addr":"172.21.15.71:6821/1556215465","heartbeat_back_addr":"172.21.15.71:6825/1556215465","heartbeat_front_addr":"172.21.15.71:6823/1556215465","state":["exists","up"]},{"osd":3,"uuid":"418fdf0b-3c97-4e70-9696-0c879202d3b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2205658375},{"type":"v1","addr":"172.21.15.71:6827","nonce":2205658375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2205658375},{"type":"v1","addr":"172.21.15.71:6829","nonce":2205658375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2205658375},{"type":"v1","addr":"172.21.15.71:6833","nonce":2205658375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2205658375},{"type":"v1","addr":"172.21.15.71:6831","nonce":2205658375}]},"public_addr":"172.21.15.71:6827/2205658375","cluster_addr":"172.21.15.71:6829/2205658375","heartbeat_back_addr":"172.21.15.71:6833/2205658375","heartbeat_front_addr":"172.21.15.71:6831/2205658375","state":["exists","up"]},{"osd":4,"uuid":"f35baaaa-6a0f-45ee-8946-1c4ad32bbd4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2383004810},{"type":"v1","addr":"172.21.15.196:6801","nonce":2383004810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2383004810},{"type":"v1","addr":"172.21.15.196:6803","nonce":2383004810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2383004810},{"type":"v1","addr":"172.21.15.196:6807","nonce":2383004810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2383004810},{"type":"v1","addr":"172.21.15.196:6805","nonce":2383004810}]},"public_addr":"172.21.15.196:6801/2383004810","cluster_addr":"172.21.15.196:6803/2383004810","heartbeat_back_addr":"172.21.15.196:6807/2383004810","heartbeat_front_addr":"172.21.15.196:6805/2383004810","state":["exists","up"]},{"osd":5,"uuid":"84bbc0da-e641-42f8-87eb-68da7f4dbd5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3314011112},{"type":"v1","addr":"172.21.15.196:6809","nonce":3314011112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3314011112},{"type":"v1","addr":"172.21.15.196:6811","nonce":3314011112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3314011112},{"type":"v1","addr":"172.21.15.196:6815","nonce":3314011112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3314011112},{"type":"v1","addr":"172.21.15.196:6813","nonce":3314011112}]},"public_addr":"172.21.15.196:6809/3314011112","cluster_addr":"172.21.15.196:6811/3314011112","heartbeat_back_addr":"172.21.15.196:6815/3314011112","heartbeat_front_addr":"172.21.15.196:6813/3314011112","state":["exists","up"]},{"osd":6,"uuid":"cff32ed5-909b-4fac-9d6d-1ea4ef27d4b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4073919265},{"type":"v1","addr":"172.21.15.196:6817","nonce":4073919265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4073919265},{"type":"v1","addr":"172.21.15.196:6819","nonce":4073919265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4073919265},{"type":"v1","addr":"172.21.15.196:6823","nonce":4073919265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4073919265},{"type":"v1","addr":"172.21.15.196:6821","nonce":4073919265}]},"public_addr":"172.21.15.196:6817/4073919265","cluster_addr":"172.21.15.196:6819/4073919265","heartbeat_back_addr":"172.21.15.196:6823/4073919265","heartbeat_front_addr":"172.21.15.196:6821/4073919265","state":["exists","up"]},{"osd":7,"uuid":"1871ecef-9cb6-4705-9c26-032088cf97c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3168217828},{"type":"v1","addr":"172.21.15.196:6825","nonce":3168217828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3168217828},{"type":"v1","addr":"172.21.15.196:6827","nonce":3168217828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3168217828},{"type":"v1","addr":"172.21.15.196:6831","nonce":3168217828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3168217828},{"type":"v1","addr":"172.21.15.196:6829","nonce":3168217828}]},"public_addr":"172.21.15.196:6825/3168217828","cluster_addr":"172.21.15.196:6827/3168217828","heartbeat_back_addr":"172.21.15.196:6831/3168217828","heartbeat_front_addr":"172.21.15.196:6829/3168217828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:08:22.552887+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:08:47.326324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:09:11.931006+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:09:36.627722+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:09:58.222107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:10:22.165516+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:10:46.332026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:11:10.482672+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6800/3468024022":"2024-09-12T08:06:59.469628+0000","172.21.15.71:0/3643762338":"2024-09-12T08:06:59.469628+0000","172.21.15.71:0/1612684168":"2024-09-12T08:06:59.469628+0000","172.21.15.71:0/3719406390":"2024-09-12T08:06:59.469628+0000","172.21.15.71:0/2967645637":"2024-09-12T08:06:29.435927+0000","172.21.15.71:6801/1255934893":"2024-09-12T08:06:29.435927+0000","172.21.15.71:6800/1255934893":"2024-09-12T08:06:29.435927+0000","172.21.15.71:6801/3468024022":"2024-09-12T08:06:59.469628+0000","172.21.15.71:0/1374934876":"2024-09-12T08:06:29.435927+0000","172.21.15.71:0/1243556948":"2024-09-12T08:06:29.435927+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:11:15.336 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:15 smithi071 ceph-mon[25834]: Detected new or changed devices on smithi196 2024-09-11T08:11:15.336 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:15 smithi071 ceph-mon[25834]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-11T08:11:15.336 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:15 smithi071 ceph-mon[25834]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:15.336 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:15 smithi071 ceph-mon[25834]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:11:15.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:15 smithi196 ceph-mon[30677]: Detected new or changed devices on smithi196 2024-09-11T08:11:15.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:15 smithi196 ceph-mon[30677]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-11T08:11:15.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:15 smithi196 ceph-mon[30677]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:15.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:15 smithi196 ceph-mon[30677]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:11:15.897 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T08:09:13.770764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-11T08:11:15.897 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T08:11:16.137 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:11:16.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:16 smithi071 ceph-mon[25834]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:11:16.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:16 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/4253994122' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:11:16.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:16 smithi071 ceph-mon[39897]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:11:16.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:16 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/4253994122' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:11:16.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:16 smithi196 ceph-mon[30677]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:11:16.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:16 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/4253994122' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:11:17.299 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:17 smithi071 ceph-mon[39897]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:17.299 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:17 smithi071 ceph-mon[39897]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:11:17.299 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:17 smithi071 ceph-mon[25834]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:17.299 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:17 smithi071 ceph-mon[25834]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:11:17.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:17 smithi196 ceph-mon[30677]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:17.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:17 smithi196 ceph-mon[30677]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:11:17.954 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-09-11T08:11:18.314 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:18 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/1385289354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:11:18.314 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:18 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/1385289354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:11:18.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:18 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/1385289354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:11:18.588 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi071 2024-09-11T08:11:18.588 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi071=foo.a' 2024-09-11T08:11:18.840 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:11:19.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:19 smithi196 ceph-mon[30677]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:19.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:19 smithi071 ceph-mon[25834]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:19.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:19 smithi071 ceph-mon[39897]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:20.495 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled rgw.foo.a update... 2024-09-11T08:11:21.243 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:21 smithi196 ceph-mon[30677]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T08:11:21.243 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:21 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:21.243 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:21 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:21.243 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:21 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:21.243 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:21 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:21.243 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:21 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:21.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:21 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:11:21.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:21 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:11:21.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:21 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:21.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:21 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:21.267 DEBUG:teuthology.orchestra.run.smithi071:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-11T08:11:21.270 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi196 2024-09-11T08:11:21.270 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-11T08:11:21.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[25834]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T08:11:21.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:21.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:21.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:21.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:21.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:21.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:11:21.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:11:21.366 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:21.366 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:21.366 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[39897]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T08:11:21.366 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:21.366 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:21.366 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:21.366 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:21.366 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:21.366 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:11:21.366 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:11:21.367 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:21.367 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:21 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:21.509 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:11:22.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:22 smithi196 ceph-mon[30677]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi071=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:22 smithi196 ceph-mon[30677]: Saving service rgw.foo.a spec with placement smithi071=foo.a;count:1 2024-09-11T08:11:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:22 smithi196 ceph-mon[30677]: Deploying daemon rgw.foo.a on smithi071 2024-09-11T08:11:22.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:22 smithi071 ceph-mon[25834]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi071=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:22.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:22 smithi071 ceph-mon[25834]: Saving service rgw.foo.a spec with placement smithi071=foo.a;count:1 2024-09-11T08:11:22.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:22 smithi071 ceph-mon[25834]: Deploying daemon rgw.foo.a on smithi071 2024-09-11T08:11:22.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:22 smithi071 ceph-mon[39897]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi071=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:22.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:22 smithi071 ceph-mon[39897]: Saving service rgw.foo.a spec with placement smithi071=foo.a;count:1 2024-09-11T08:11:22.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:22 smithi071 ceph-mon[39897]: Deploying daemon rgw.foo.a on smithi071 2024-09-11T08:11:23.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:23 smithi196 ceph-mon[30677]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-11T08:11:23.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:23 smithi071 ceph-mon[25834]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-11T08:11:23.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:23 smithi071 ceph-mon[39897]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-11T08:11:23.195 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:11:23 smithi071 podman[77004]: 2024-09-11 08:11:23.138836357 +0000 UTC m=+0.128830302 container create 3f1ac611d9b91c2002dbcc8138686a44db29328b7f84b676e097661590991df2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-rgw-foo-a, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:11:23.695 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:11:23 smithi071 podman[77004]: 2024-09-11 08:11:23.357692442 +0000 UTC m=+0.347686341 container init 3f1ac611d9b91c2002dbcc8138686a44db29328b7f84b676e097661590991df2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-11T08:11:23.695 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:11:23 smithi071 podman[77004]: 2024-09-11 08:11:23.362662408 +0000 UTC m=+0.352656307 container start 3f1ac611d9b91c2002dbcc8138686a44db29328b7f84b676e097661590991df2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-09-11T08:11:23.695 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:11:23 smithi071 bash[77004]: 3f1ac611d9b91c2002dbcc8138686a44db29328b7f84b676e097661590991df2 2024-09-11T08:11:23.695 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:11:23 smithi071 systemd[1]: Started Ceph rgw.foo.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:11:24.078 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'datapool' created 2024-09-11T08:11:24.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:24 smithi196 ceph-mon[30677]: from='client.? 172.21.15.196:0/3650258547' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:11:24.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:24 smithi196 ceph-mon[30677]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:11:24.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:24 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:24.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:24 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:24.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:24 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:24.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:24 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:24.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:24 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:24.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:24 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:24.436 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:24 smithi071 ceph-mon[25834]: from='client.? 172.21.15.196:0/3650258547' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:11:24.436 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:24 smithi071 ceph-mon[25834]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:11:24.436 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:24 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:24.436 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:24 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:24.436 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:24 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:24.436 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:24 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:24.436 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:24 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:24.436 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:24 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:24.436 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:24 smithi071 ceph-mon[39897]: from='client.? 172.21.15.196:0/3650258547' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:11:24.437 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:24 smithi071 ceph-mon[39897]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:11:24.437 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:24 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:24.437 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:24 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:24.437 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:24 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:24.437 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:24 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:24.437 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:24 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:24.437 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:24 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:24.690 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-11T08:11:24.929 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:11:25.375 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:25 smithi196 ceph-mon[30677]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-11T08:11:25.375 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:25 smithi196 ceph-mon[30677]: Saving service rgw.foo.a spec with placement smithi071=foo.a;count:1 2024-09-11T08:11:25.375 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:25 smithi196 ceph-mon[30677]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:11:25.375 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:25 smithi196 ceph-mon[30677]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:11:25.375 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:25 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/2859568968' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:11:25.376 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:25 smithi196 ceph-mon[30677]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:11:25.376 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:25 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:25.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:25 smithi071 ceph-mon[39897]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-11T08:11:25.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:25 smithi071 ceph-mon[39897]: Saving service rgw.foo.a spec with placement smithi071=foo.a;count:1 2024-09-11T08:11:25.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:25 smithi071 ceph-mon[39897]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:11:25.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:25 smithi071 ceph-mon[39897]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:11:25.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:25 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/2859568968' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:11:25.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:25 smithi071 ceph-mon[39897]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:11:25.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:25 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:25.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:25 smithi071 ceph-mon[25834]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-11T08:11:25.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:25 smithi071 ceph-mon[25834]: Saving service rgw.foo.a spec with placement smithi071=foo.a;count:1 2024-09-11T08:11:25.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:25 smithi071 ceph-mon[25834]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:11:25.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:25 smithi071 ceph-mon[25834]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:11:25.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:25 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2859568968' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:11:25.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:25 smithi071 ceph-mon[25834]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:11:25.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:25 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:26.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:26 smithi196 ceph-mon[30677]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:11:26.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:26 smithi196 ceph-mon[30677]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:11:26.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:26 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:26.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:26 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:26.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:26 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:26.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:26 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:26.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:26 smithi196 ceph-mon[30677]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:11:26.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:26 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/2859568968' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:11:26.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:26 smithi196 ceph-mon[30677]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:11:26.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:26 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/3351423616' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:11:26.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[25834]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:11:26.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[25834]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:11:26.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:26.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:26.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:26.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:26.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[25834]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:11:26.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2859568968' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:11:26.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[25834]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:11:26.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3351423616' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:11:26.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[39897]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:11:26.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[39897]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:11:26.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:26.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:26.447 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:26.447 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:26.447 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[39897]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:11:26.447 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/2859568968' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:11:26.447 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[39897]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:11:26.447 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:26 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/3351423616' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:11:27.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:27 smithi196 ceph-mon[30677]: Checking dashboard <-> RGW credentials 2024-09-11T08:11:27.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:27 smithi196 ceph-mon[30677]: pgmap v183: 36 pgs: 12 creating+peering, 20 unknown, 4 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:11:27.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:27 smithi196 ceph-mon[30677]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:27.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:27 smithi196 ceph-mon[30677]: from='client.? 172.21.15.196:0/4013543867' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:11:27.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:27 smithi196 ceph-mon[30677]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:11:27.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:27 smithi196 ceph-mon[30677]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:11:27.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:27 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/3351423616' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:11:27.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:27 smithi196 ceph-mon[30677]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:11:27.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:27 smithi196 ceph-mon[30677]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:11:27.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[39897]: Checking dashboard <-> RGW credentials 2024-09-11T08:11:27.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[39897]: pgmap v183: 36 pgs: 12 creating+peering, 20 unknown, 4 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:11:27.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[39897]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:27.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[39897]: from='client.? 172.21.15.196:0/4013543867' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:11:27.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[39897]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:11:27.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[39897]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:11:27.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/3351423616' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:11:27.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[39897]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:11:27.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[39897]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:11:27.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[25834]: Checking dashboard <-> RGW credentials 2024-09-11T08:11:27.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[25834]: pgmap v183: 36 pgs: 12 creating+peering, 20 unknown, 4 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:11:27.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[25834]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:27.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[25834]: from='client.? 172.21.15.196:0/4013543867' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:11:27.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[25834]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:11:27.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[25834]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:11:27.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3351423616' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:11:27.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[25834]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:11:27.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:27 smithi071 ceph-mon[25834]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:11:29.419 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:29 smithi196 ceph-mon[30677]: pgmap v186: 68 pgs: 12 creating+peering, 46 unknown, 10 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:11:29.419 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:29 smithi196 ceph-mon[30677]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:11:29.419 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:29 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/2859568968' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:11:29.419 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:29 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/3351423616' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:11:29.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:29 smithi196 ceph-mon[30677]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:11:29.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:29 smithi196 ceph-mon[30677]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:11:29.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:29 smithi196 ceph-mon[30677]: Cluster is now healthy 2024-09-11T08:11:29.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:29 smithi071 ceph-mon[25834]: pgmap v186: 68 pgs: 12 creating+peering, 46 unknown, 10 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:11:29.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:29 smithi071 ceph-mon[25834]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:11:29.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:29 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2859568968' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:11:29.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:29 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3351423616' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:11:29.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:29 smithi071 ceph-mon[25834]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:11:29.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:29 smithi071 ceph-mon[25834]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:11:29.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:29 smithi071 ceph-mon[25834]: Cluster is now healthy 2024-09-11T08:11:29.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:29 smithi071 ceph-mon[39897]: pgmap v186: 68 pgs: 12 creating+peering, 46 unknown, 10 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:11:29.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:29 smithi071 ceph-mon[39897]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:11:29.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:29 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/2859568968' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:11:29.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:29 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/3351423616' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:11:29.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:29 smithi071 ceph-mon[39897]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:11:29.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:29 smithi071 ceph-mon[39897]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:11:29.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:29 smithi071 ceph-mon[39897]: Cluster is now healthy 2024-09-11T08:11:29.730 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.196 --placement '1;smithi196=iscsi.a' 2024-09-11T08:11:29.969 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:11:30.339 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:30 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/3351423616' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:11:30.339 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:30 smithi196 ceph-mon[30677]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:11:30.339 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:30 smithi196 ceph-mon[30677]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:11:30.420 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:30 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3351423616' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:11:30.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:30 smithi071 ceph-mon[25834]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:11:30.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:30 smithi071 ceph-mon[25834]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:11:30.421 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:30 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/3351423616' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:11:30.421 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:30 smithi071 ceph-mon[39897]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:11:30.421 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:30 smithi071 ceph-mon[39897]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:11:31.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:31 smithi196 ceph-mon[30677]: pgmap v189: 100 pgs: 20 creating+peering, 45 unknown, 35 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-11T08:11:31.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:31 smithi196 ceph-mon[30677]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:11:31.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:31 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/737078157' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:11:31.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:31 smithi196 ceph-mon[30677]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:11:31.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:31 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/3154699838' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:11:31.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:31 smithi196 ceph-mon[30677]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:11:31.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:31 smithi071 ceph-mon[25834]: pgmap v189: 100 pgs: 20 creating+peering, 45 unknown, 35 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-11T08:11:31.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:31 smithi071 ceph-mon[25834]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:11:31.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:31 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/737078157' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:11:31.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:31 smithi071 ceph-mon[25834]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:11:31.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:31 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3154699838' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:11:31.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:31 smithi071 ceph-mon[25834]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:11:31.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:31 smithi071 ceph-mon[39897]: pgmap v189: 100 pgs: 20 creating+peering, 45 unknown, 35 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-11T08:11:31.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:31 smithi071 ceph-mon[39897]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:11:31.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:31 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/737078157' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:11:31.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:31 smithi071 ceph-mon[39897]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:11:31.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:31 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/3154699838' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:11:31.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:31 smithi071 ceph-mon[39897]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:11:31.825 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled iscsi.datapool update... 2024-09-11T08:11:32.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:32 smithi196 ceph-mon[30677]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:11:32.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:32 smithi196 ceph-mon[30677]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:11:32.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:32 smithi196 ceph-mon[30677]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:11:32.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:32 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/3154699838' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:11:32.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:32 smithi196 ceph-mon[30677]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:11:32.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:32 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/737078157' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:11:32.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:32 smithi196 ceph-mon[30677]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:11:32.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:32 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:32.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:32 smithi196 ceph-mon[30677]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:11:32.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:32 smithi196 ceph-mon[30677]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:11:32.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:32 smithi196 ceph-mon[30677]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T08:11:32.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[39897]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:11:32.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[39897]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:11:32.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[39897]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:11:32.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/3154699838' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:11:32.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[39897]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:11:32.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/737078157' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:11:32.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[39897]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:11:32.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:32.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[39897]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:11:32.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[39897]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:11:32.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[39897]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T08:11:32.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[25834]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:11:32.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[25834]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:11:32.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[25834]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:11:32.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3154699838' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:11:32.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[25834]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:11:32.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/737078157' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:11:32.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[25834]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:11:32.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:32.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[25834]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:11:32.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[25834]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:11:32.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:32 smithi071 ceph-mon[25834]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T08:11:32.545 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-11T08:11:32.545 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T08:11:32.546 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T08:11:32.575 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T08:11:32.575 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T08:11:32.605 DEBUG:teuthology.orchestra.run.smithi196:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-11T08:11:32.648 INFO:tasks.cephadm:Adding prometheus.a on smithi196 2024-09-11T08:11:32.648 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi196=a' 2024-09-11T08:11:33.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:33 smithi196 ceph-mon[30677]: pgmap v192: 132 pgs: 18 creating+peering, 28 unknown, 86 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-11T08:11:33.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:33 smithi196 ceph-mon[30677]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.196", "placement": "1;smithi196=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:33.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:33 smithi196 ceph-mon[30677]: Saving service iscsi.datapool spec with placement smithi196=iscsi.a;count:1 2024-09-11T08:11:33.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:33 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:33.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:33 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:33.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:33 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:33.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:33 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:11:33.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:33 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:11:33.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:33 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:33.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:33 smithi196 ceph-mon[30677]: Deploying daemon iscsi.iscsi.a on smithi196 2024-09-11T08:11:33.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[25834]: pgmap v192: 132 pgs: 18 creating+peering, 28 unknown, 86 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-11T08:11:33.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[25834]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.196", "placement": "1;smithi196=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:33.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[25834]: Saving service iscsi.datapool spec with placement smithi196=iscsi.a;count:1 2024-09-11T08:11:33.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:33.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:33.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:33.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:11:33.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:11:33.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:33.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[25834]: Deploying daemon iscsi.iscsi.a on smithi196 2024-09-11T08:11:33.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[39897]: pgmap v192: 132 pgs: 18 creating+peering, 28 unknown, 86 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-11T08:11:33.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[39897]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.196", "placement": "1;smithi196=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:33.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[39897]: Saving service iscsi.datapool spec with placement smithi196=iscsi.a;count:1 2024-09-11T08:11:33.446 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:33.447 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:33.447 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:33.447 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:11:33.447 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:11:33.447 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:33.447 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:33 smithi071 ceph-mon[39897]: Deploying daemon iscsi.iscsi.a on smithi196 2024-09-11T08:11:33.683 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:11:34.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:34 smithi196 ceph-mon[30677]: pgmap v194: 132 pgs: 18 creating+peering, 18 unknown, 96 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-11T08:11:34.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:34 smithi071 ceph-mon[25834]: pgmap v194: 132 pgs: 18 creating+peering, 18 unknown, 96 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-11T08:11:34.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:34 smithi071 ceph-mon[39897]: pgmap v194: 132 pgs: 18 creating+peering, 18 unknown, 96 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-11T08:11:35.668 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled prometheus update... 2024-09-11T08:11:36.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:35 smithi196 conmon[68390]: conmon 6633cb4cfcf363ca313a : Failed to add inotify watch for /sys/fs/cgroup/system.slice/system-ceph\x2d566367e6\x2d7014\x2d11ef\x2dbcea\x2dc7b262605968.slice/ceph-566367e6-7014-11ef-bcea-c7b262605968@iscsi.iscsi.a.service/libpod-payload-6633cb4cfcf363ca313aad30c9fb66f6a96ea3ecc589472ff2a5887abbec938c/memory.events 2024-09-11T08:11:36.640 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:36 smithi196 podman[68290]: 2024-09-11 08:11:36.29513217 +0000 UTC m=+1.401688707 container init 6633cb4cfcf363ca313aad30c9fb66f6a96ea3ecc589472ff2a5887abbec938c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T08:11:36.640 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:36 smithi196 podman[68290]: 2024-09-11 08:11:36.300415558 +0000 UTC m=+1.406972096 container start 6633cb4cfcf363ca313aad30c9fb66f6a96ea3ecc589472ff2a5887abbec938c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=) 2024-09-11T08:11:36.640 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:36 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu[68390]: log file path now is '/var/log/tcmu-runner.log' 2024-09-11T08:11:36.640 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:36 smithi196 bash[68290]: 6633cb4cfcf363ca313aad30c9fb66f6a96ea3ecc589472ff2a5887abbec938c 2024-09-11T08:11:36.914 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:36 smithi196 ceph-mon[30677]: pgmap v195: 132 pgs: 5 creating+peering, 127 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s rd, 5.2 KiB/s wr, 157 op/s 2024-09-11T08:11:36.914 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:36 smithi196 ceph-mon[30677]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:36.914 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:36 smithi196 ceph-mon[30677]: Saving service prometheus spec with placement smithi196=a;count:1 2024-09-11T08:11:36.915 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:36 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:36.941 DEBUG:teuthology.orchestra.run.smithi196:prometheus.a> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-11T08:11:36.944 INFO:tasks.cephadm:Adding node-exporter.a on smithi071 2024-09-11T08:11:36.944 INFO:tasks.cephadm:Adding node-exporter.b on smithi196 2024-09-11T08:11:36.944 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi071=a;smithi196=b' 2024-09-11T08:11:36.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:36 smithi071 ceph-mon[39897]: pgmap v195: 132 pgs: 5 creating+peering, 127 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s rd, 5.2 KiB/s wr, 157 op/s 2024-09-11T08:11:36.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:36 smithi071 ceph-mon[39897]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:36.947 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:36 smithi071 ceph-mon[39897]: Saving service prometheus spec with placement smithi196=a;count:1 2024-09-11T08:11:36.947 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:36 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:36.947 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:36 smithi071 ceph-mon[25834]: pgmap v195: 132 pgs: 5 creating+peering, 127 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s rd, 5.2 KiB/s wr, 157 op/s 2024-09-11T08:11:36.947 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:36 smithi071 ceph-mon[25834]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:36.947 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:36 smithi071 ceph-mon[25834]: Saving service prometheus spec with placement smithi196=a;count:1 2024-09-11T08:11:36.947 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:36 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:37.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 podman[68448]: 2024-09-11 08:11:36.972656091 +0000 UTC m=+0.449658432 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:11:37.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 podman[68448]: 2024-09-11 08:11:37.089994436 +0000 UTC m=+0.566996779 container create da81cc55fa16f4f0a5babbacd884c2af3e664af7b305b0e108998040693549e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T08:11:37.468 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:11:37.598 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 podman[68448]: 2024-09-11 08:11:37.342656 +0000 UTC m=+0.819658348 container init da81cc55fa16f4f0a5babbacd884c2af3e664af7b305b0e108998040693549e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:11:37.598 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 podman[68448]: 2024-09-11 08:11:37.348102755 +0000 UTC m=+0.825105095 container start da81cc55fa16f4f0a5babbacd884c2af3e664af7b305b0e108998040693549e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-11T08:11:37.598 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 bash[68448]: da81cc55fa16f4f0a5babbacd884c2af3e664af7b305b0e108998040693549e5 2024-09-11T08:11:37.598 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 systemd[1]: Started Ceph iscsi.iscsi.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:11:37.873 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug Started the configuration object watcher 2024-09-11T08:11:37.874 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug Checking for config object changes every 1s 2024-09-11T08:11:37.874 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug Processing osd blocklist entries for this node 2024-09-11T08:11:38.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug Reading the configuration object to update local LIO configuration 2024-09-11T08:11:38.182 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug Configuration does not have an entry for this host(smithi196.front.sepia.ceph.com) - nothing to define to LIO 2024-09-11T08:11:38.182 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-11T08:11:38.182 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: * Environment: production 2024-09-11T08:11:38.182 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:11:38.182 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: Use a production WSGI server instead. 2024-09-11T08:11:38.182 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: * Debug mode: off 2024-09-11T08:11:38.182 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug * Running on all addresses. 2024-09-11T08:11:38.182 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:11:38.183 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: * Running on all addresses. 2024-09-11T08:11:38.183 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:11:38.183 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug * Running on http://[fe80::ec4:7aff:fe88:7f47]:5000/ (Press CTRL+C to quit) 2024-09-11T08:11:38.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: * Running on http://[fe80::ec4:7aff:fe88:7f47]:5000/ (Press CTRL+C to quit) 2024-09-11T08:11:38.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:38 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:38.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:38 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:38.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:38 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:38.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:38 smithi196 ceph-mon[30677]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:11:38.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:38 smithi196 ceph-mon[30677]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 62 KiB/s rd, 4.8 KiB/s wr, 150 op/s 2024-09-11T08:11:38.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:38 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:38.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:38 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:38.626 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:38 smithi196 ceph-mon[30677]: from='client.? 172.21.15.196:0/1972424495' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:11:38.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:38 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:38.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:38 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:38.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:38 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:38.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:38 smithi071 ceph-mon[25834]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:11:38.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:38 smithi071 ceph-mon[25834]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 62 KiB/s rd, 4.8 KiB/s wr, 150 op/s 2024-09-11T08:11:38.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:38 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:38.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:38 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:38.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:38 smithi071 ceph-mon[25834]: from='client.? 172.21.15.196:0/1972424495' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:11:38.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:38 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:38.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:38 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:38.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:38 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:38.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:38 smithi071 ceph-mon[39897]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:11:38.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:38 smithi071 ceph-mon[39897]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 62 KiB/s rd, 4.8 KiB/s wr, 150 op/s 2024-09-11T08:11:38.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:38 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:38.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:38 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:38.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:38 smithi071 ceph-mon[39897]: from='client.? 172.21.15.196:0/1972424495' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:11:39.309 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled node-exporter update... 2024-09-11T08:11:39.750 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:39 smithi196 ceph-mon[30677]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:11:39.751 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:39 smithi196 ceph-mon[30677]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T08:11:39.751 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:39 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:39.924 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:39 smithi071 ceph-mon[39897]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:11:39.924 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:39 smithi071 ceph-mon[39897]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T08:11:39.924 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:39 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:39.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:39 smithi071 ceph-mon[25834]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:11:39.925 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:39 smithi071 ceph-mon[25834]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T08:11:39.925 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:39 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:40.418 DEBUG:teuthology.orchestra.run.smithi071:node-exporter.a> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-11T08:11:40.421 DEBUG:teuthology.orchestra.run.smithi196:node-exporter.b> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-11T08:11:40.424 INFO:tasks.cephadm:Adding alertmanager.a on smithi071 2024-09-11T08:11:40.424 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi071=a' 2024-09-11T08:11:40.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:40 smithi196 ceph-mon[30677]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi071=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:40 smithi196 ceph-mon[30677]: Saving service node-exporter spec with placement smithi071=a;smithi196=b;count:2 2024-09-11T08:11:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:40 smithi196 ceph-mon[30677]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 57 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-09-11T08:11:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:40 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:40.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:40 smithi071 ceph-mon[25834]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi071=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:40.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:40 smithi071 ceph-mon[25834]: Saving service node-exporter spec with placement smithi071=a;smithi196=b;count:2 2024-09-11T08:11:40.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:40 smithi071 ceph-mon[25834]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 57 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-09-11T08:11:40.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:40 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:40 smithi071 ceph-mon[39897]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi071=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:40 smithi071 ceph-mon[39897]: Saving service node-exporter spec with placement smithi071=a;smithi196=b;count:2 2024-09-11T08:11:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:40 smithi071 ceph-mon[39897]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 57 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-09-11T08:11:40.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:40 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:41.429 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:11:42.585 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:42 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:42.585 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:42 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:42.585 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:42 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:42.585 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:42 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:42.585 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:42 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:42.585 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:42 smithi196 ceph-mon[30677]: Deploying daemon prometheus.a on smithi196 2024-09-11T08:11:42.585 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:42 smithi196 ceph-mon[30677]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 3.8 KiB/s wr, 119 op/s 2024-09-11T08:11:42.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:42 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:42.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:42 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:42.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:42 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:42.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:42 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:42.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:42 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:42.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:42 smithi071 ceph-mon[25834]: Deploying daemon prometheus.a on smithi196 2024-09-11T08:11:42.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:42 smithi071 ceph-mon[25834]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 3.8 KiB/s wr, 119 op/s 2024-09-11T08:11:42.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:42.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:42.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:42.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:42.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:42 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:42.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:42 smithi071 ceph-mon[39897]: Deploying daemon prometheus.a on smithi196 2024-09-11T08:11:42.946 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:42 smithi071 ceph-mon[39897]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 3.8 KiB/s wr, 119 op/s 2024-09-11T08:11:43.246 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled alertmanager update... 2024-09-11T08:11:44.171 DEBUG:teuthology.orchestra.run.smithi071:alertmanager.a> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-11T08:11:44.174 INFO:tasks.cephadm:Adding grafana.a on smithi196 2024-09-11T08:11:44.174 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi196=a' 2024-09-11T08:11:44.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:44 smithi196 ceph-mon[30677]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi071=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:44.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:44 smithi196 ceph-mon[30677]: Saving service alertmanager spec with placement smithi071=a;count:1 2024-09-11T08:11:44.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:44 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:44.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:44 smithi196 ceph-mon[30677]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 46 KiB/s rd, 3.4 KiB/s wr, 112 op/s 2024-09-11T08:11:44.620 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:11:44.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:44 smithi071 ceph-mon[25834]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi071=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:44.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:44 smithi071 ceph-mon[25834]: Saving service alertmanager spec with placement smithi071=a;count:1 2024-09-11T08:11:44.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:44 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:44.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:44 smithi071 ceph-mon[25834]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 46 KiB/s rd, 3.4 KiB/s wr, 112 op/s 2024-09-11T08:11:44.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:44 smithi071 ceph-mon[39897]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi071=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:44.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:44 smithi071 ceph-mon[39897]: Saving service alertmanager spec with placement smithi071=a;count:1 2024-09-11T08:11:44.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:44 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:44.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:44 smithi071 ceph-mon[39897]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 46 KiB/s rd, 3.4 KiB/s wr, 112 op/s 2024-09-11T08:11:45.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:45 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:45.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:45 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:45.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:45 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:46.666 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:46 smithi196 ceph-mon[30677]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.7 KiB/s rd, 614 B/s wr, 18 op/s 2024-09-11T08:11:46.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:46 smithi071 ceph-mon[25834]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.7 KiB/s rd, 614 B/s wr, 18 op/s 2024-09-11T08:11:46.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:46 smithi071 ceph-mon[39897]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.7 KiB/s rd, 614 B/s wr, 18 op/s 2024-09-11T08:11:47.235 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled grafana update... 2024-09-11T08:11:48.059 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:11:47 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug there is no tcmu-runner data available 2024-09-11T08:11:48.087 DEBUG:teuthology.orchestra.run.smithi196:grafana.a> sudo journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@grafana.a.service 2024-09-11T08:11:48.089 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T08:11:48.089 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T08:11:48.342 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:11:48.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:48 smithi196 ceph-mon[30677]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:48.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:48 smithi196 ceph-mon[30677]: Saving service grafana spec with placement smithi196=a;count:1 2024-09-11T08:11:48.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:48 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:48.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:48 smithi196 ceph-mon[30677]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 614 B/s rd, 102 B/s wr, 1 op/s 2024-09-11T08:11:48.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:48 smithi196 ceph-mon[30677]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:48.444 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:48 smithi071 ceph-mon[25834]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:48.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:48 smithi071 ceph-mon[25834]: Saving service grafana spec with placement smithi196=a;count:1 2024-09-11T08:11:48.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:48 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:48.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:48 smithi071 ceph-mon[25834]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 614 B/s rd, 102 B/s wr, 1 op/s 2024-09-11T08:11:48.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:48 smithi071 ceph-mon[25834]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:48.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:48 smithi071 ceph-mon[39897]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:11:48.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:48 smithi071 ceph-mon[39897]: Saving service grafana spec with placement smithi196=a;count:1 2024-09-11T08:11:48.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:48 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:48.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:48 smithi071 ceph-mon[39897]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 614 B/s rd, 102 B/s wr, 1 op/s 2024-09-11T08:11:48.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:48 smithi071 ceph-mon[39897]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:50.457 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-09-11T08:11:50.457 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQDGUOFmfV0CGxAA1LAY9m23rAE28zmQeA5/PQ== 2024-09-11T08:11:50.823 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:50 smithi071 ceph-mon[39897]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 954 B/s rd, 0 op/s 2024-09-11T08:11:50.823 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:50 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/1265114249' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:11:50.823 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:50 smithi071 ceph-mon[39897]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:11:50.824 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:50 smithi071 ceph-mon[39897]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:11:50.824 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:50 smithi071 ceph-mon[25834]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 954 B/s rd, 0 op/s 2024-09-11T08:11:50.824 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:50 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/1265114249' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:11:50.824 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:50 smithi071 ceph-mon[25834]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:11:50.824 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:50 smithi071 ceph-mon[25834]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:11:50.847 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:50 smithi196 ceph-mon[30677]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 954 B/s rd, 0 op/s 2024-09-11T08:11:50.848 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:50 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/1265114249' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:11:50.848 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:50 smithi196 ceph-mon[30677]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:11:50.848 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:50 smithi196 ceph-mon[30677]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:11:51.145 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T08:11:51.146 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T08:11:51.146 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T08:11:51.183 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T08:11:52.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:52 smithi196 ceph-mon[30677]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:52.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:52 smithi071 ceph-mon[25834]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:52.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:52 smithi071 ceph-mon[39897]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:53.310 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:11:54.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:54 smithi196 ceph-mon[30677]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:54.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:54 smithi071 ceph-mon[39897]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:54.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:54 smithi071 ceph-mon[25834]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:54.984 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:54 smithi196 podman[70326]: 2024-09-11 08:11:54.862268438 +0000 UTC m=+0.143554117 container create 78dd4300077191a963615b3d717b299efe268ab87f3ae7bb65adf944afc17d31 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:11:55.339 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 podman[70326]: 2024-09-11 08:11:55.333728025 +0000 UTC m=+0.615013658 container init 78dd4300077191a963615b3d717b299efe268ab87f3ae7bb65adf944afc17d31 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:11:55.340 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 podman[70326]: 2024-09-11 08:11:55.338997126 +0000 UTC m=+0.620282767 container start 78dd4300077191a963615b3d717b299efe268ab87f3ae7bb65adf944afc17d31 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:11:55.592 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.363Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T08:11:55.592 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.363Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T08:11:55.592 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.363Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi196 (none))" 2024-09-11T08:11:55.592 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.363Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T08:11:55.592 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.363Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T08:11:55.592 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.364Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T08:11:55.592 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.364Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T08:11:55.592 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.365Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T08:11:55.592 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.365Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T08:11:55.592 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.367Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T08:11:55.592 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.367Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.68µs 2024-09-11T08:11:55.593 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.367Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T08:11:55.593 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.368Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-11T08:11:55.593 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.368Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=23.581µs wal_replay_duration=254.279µs wbl_replay_duration=105ns total_replay_duration=296.146µs 2024-09-11T08:11:55.593 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.368Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T08:11:55.593 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.368Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T08:11:55.593 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.368Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T08:11:55.593 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.382Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=13.54979ms db_storage=804ns remote_storage=974ns web_handler=376ns query_engine=597ns scrape=390.042µs scrape_sd=18.747µs notify=761ns notify_sd=1.395µs rules=12.951297ms tracing=6.096µs 2024-09-11T08:11:55.593 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.382Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T08:11:55.593 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:11:55.382Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T08:11:55.593 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 bash[70326]: 78dd4300077191a963615b3d717b299efe268ab87f3ae7bb65adf944afc17d31 2024-09-11T08:11:55.593 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:11:55 smithi196 systemd[1]: Started Ceph prometheus.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:11:55.898 INFO:teuthology.orchestra.run.smithi196.stdout:[client.1] 2024-09-11T08:11:55.898 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDLUOFmslgrNRAArAtb9ITD9Bf0/r3Z3ClGcA== 2024-09-11T08:11:56.556 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:56.556 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:56.556 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:56.556 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:56 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:11:56.556 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:56 smithi196 ceph-mon[30677]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:56.556 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:56 smithi196 ceph-mon[30677]: from='client.? 172.21.15.196:0/1739435597' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:11:56.556 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:56 smithi196 ceph-mon[30677]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:11:56.556 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:56 smithi196 ceph-mon[30677]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:11:56.608 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T08:11:56.608 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-11T08:11:56.608 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-11T08:11:56.646 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T08:11:56.647 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T08:11:56.647 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T08:11:56.793 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: ignoring --setuser ceph since I am not root 2024-09-11T08:11:56.793 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: ignoring --setgroup ceph since I am not root 2024-09-11T08:11:56.793 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mgr[26112]: -- 172.21.15.71:0/2153060700 <== mon.2 v2:172.21.15.71:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555fedf62340 con 0x555fedf6ec00 2024-09-11T08:11:56.793 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:56.667+0000 7f81c1616ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:11:56.793 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:56.724+0000 7f81c1616ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:11:56.794 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:56.794 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:56.794 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:56.794 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:11:56.794 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mon[25834]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:56.794 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mon[25834]: from='client.? 172.21.15.196:0/1739435597' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:11:56.794 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mon[25834]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:11:56.794 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mon[25834]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:11:56.794 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:56.795 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:56.795 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' 2024-09-11T08:11:56.795 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:11:56.795 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mon[39897]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:11:56.795 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mon[39897]: from='client.? 172.21.15.196:0/1739435597' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:11:56.795 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mon[39897]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:11:56.795 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-mon[39897]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:11:56.898 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:11:56.907 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:56 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: ignoring --setuser ceph since I am not root 2024-09-11T08:11:56.907 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:56 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: ignoring --setgroup ceph since I am not root 2024-09-11T08:11:56.908 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:56 smithi196 ceph-mgr[32783]: -- 172.21.15.196:0/2651388939 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cb2e79c340 con 0x55cb2e7a8c00 2024-09-11T08:11:56.908 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:56 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:56.667+0000 7ffa77f01ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:11:56.908 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:56 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:56.721+0000 7ffa77f01ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:11:56.908 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:56 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:56.785+0000 7ffa77f01ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:11:56.908 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:56 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:56.906+0000 7ffa77f01ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:11:57.049 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:56.792+0000 7f81c1616ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:11:57.049 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:56 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:56.930+0000 7f81c1616ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:11:57.181 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:57 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:57.040+0000 7ffa77f01ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:11:57.318 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:57 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:57.069+0000 7f81c1616ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:11:57.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:57 smithi196 ceph-mon[30677]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:11:57.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:57 smithi196 ceph-mon[30677]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:11:57.681 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:57 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:57.294+0000 7ffa77f01ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:11:57.681 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:57 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:57.353+0000 7ffa77f01ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:11:57.681 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:57 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:57.511+0000 7ffa77f01ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:11:57.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:57 smithi071 ceph-mon[39897]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:11:57.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:57 smithi071 ceph-mon[39897]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:11:57.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:57 smithi071 ceph-mon[25834]: from='mgr.14150 172.21.15.71:0/1365660426' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:11:57.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:57 smithi071 ceph-mon[25834]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:11:57.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:57 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:57.336+0000 7f81c1616ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:11:57.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:57 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:57.397+0000 7f81c1616ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:11:57.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:57 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:57.560+0000 7f81c1616ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:11:58.681 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:58 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:58.339+0000 7ffa77f01ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:11:58.681 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:58 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:58.402+0000 7ffa77f01ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:11:58.692 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:58 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:58.412+0000 7f81c1616ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:11:58.692 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:58 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:58.482+0000 7f81c1616ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:11:58.880 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:11:58.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:11:58 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/640243600' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:11:58.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:11:58 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/640243600' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:11:59.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:11:58 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/640243600' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:11:59.181 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:59.106+0000 7ffa77f01ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:11:59.513 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:59 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:59.217+0000 7f81c1616ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:11:59.513 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:59 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:59.330+0000 7f81c1616ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:11:59.513 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:59 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:59.451+0000 7f81c1616ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:11:59.513 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:59 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:59.511+0000 7f81c1616ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:11:59.518 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:59.217+0000 7ffa77f01ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:11:59.519 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:59.335+0000 7ffa77f01ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:11:59.519 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:59.394+0000 7ffa77f01ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:11:59.519 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:59.517+0000 7ffa77f01ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:11:59.595 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":1121251349},{"type":"v1","addr":"172.21.15.71:6801","nonce":1121251349}]},"active_addr":"172.21.15.71:6801/1121251349","active_change":"2024-09-11T08:06:59.469766+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24109,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.71:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":799944144}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":2294552436}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":2106434609}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":1292894177}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":2259311326}]}]} 2024-09-11T08:11:59.598 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T08:11:59.598 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T08:11:59.599 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:11:59.774 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:59.584+0000 7ffa77f01ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:11:59.775 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:59.641+0000 7ffa77f01ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:11:59.775 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:59.773+0000 7ffa77f01ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:11:59.776 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:59 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:59.643+0000 7f81c1616ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:11:59.776 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:59 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:59.714+0000 7f81c1616ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:11:59.776 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:59 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:59.775+0000 7f81c1616ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:11:59.853 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:00.039 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:59.836+0000 7ffa77f01ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:12:00.040 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:59.901+0000 7ffa77f01ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:12:00.040 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:11:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:11:59.981+0000 7ffa77f01ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:12:00.040 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:12:00 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:12:00.038+0000 7ffa77f01ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:12:00.057 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:59 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:59.927+0000 7f81c1616ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:12:00.057 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:11:59 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:11:59.990+0000 7f81c1616ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:12:00.057 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:12:00.055+0000 7f81c1616ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:12:00.358 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:12:00.142+0000 7f81c1616ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:12:00.358 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:12:00.203+0000 7f81c1616ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:12:00.358 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:00 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:12:00.261+0000 7f81c1616ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:12:00.431 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:12:00 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:12:00.095+0000 7ffa77f01ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:12:01.049 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:00 smithi071 ceph-mon[25834]: Standby manager daemon x restarted 2024-09-11T08:12:01.049 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:00 smithi071 ceph-mon[25834]: Standby manager daemon x started 2024-09-11T08:12:01.049 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:00 smithi071 ceph-mon[25834]: from='mgr.? 172.21.15.196:0/1303182838' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:12:01.049 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:00 smithi071 ceph-mon[25834]: from='mgr.? 172.21.15.196:0/1303182838' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:12:01.049 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:00 smithi071 ceph-mon[25834]: from='mgr.? 172.21.15.196:0/1303182838' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:12:01.049 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:00 smithi071 ceph-mon[25834]: from='mgr.? 172.21.15.196:0/1303182838' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:12:01.050 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:00 smithi071 ceph-mon[39897]: Standby manager daemon x restarted 2024-09-11T08:12:01.050 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:00 smithi071 ceph-mon[39897]: Standby manager daemon x started 2024-09-11T08:12:01.050 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:00 smithi071 ceph-mon[39897]: from='mgr.? 172.21.15.196:0/1303182838' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:12:01.050 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:00 smithi071 ceph-mon[39897]: from='mgr.? 172.21.15.196:0/1303182838' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:12:01.050 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:00 smithi071 ceph-mon[39897]: from='mgr.? 172.21.15.196:0/1303182838' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:12:01.050 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:00 smithi071 ceph-mon[39897]: from='mgr.? 172.21.15.196:0/1303182838' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:12:01.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:00 smithi196 ceph-mon[30677]: Standby manager daemon x restarted 2024-09-11T08:12:01.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:00 smithi196 ceph-mon[30677]: Standby manager daemon x started 2024-09-11T08:12:01.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:00 smithi196 ceph-mon[30677]: from='mgr.? 172.21.15.196:0/1303182838' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:12:01.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:00 smithi196 ceph-mon[30677]: from='mgr.? 172.21.15.196:0/1303182838' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:12:01.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:00 smithi196 ceph-mon[30677]: from='mgr.? 172.21.15.196:0/1303182838' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:12:01.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:00 smithi196 ceph-mon[30677]: from='mgr.? 172.21.15.196:0/1303182838' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:12:01.182 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:12:00 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:12:00.832+0000 7ffa77f01ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:12:01.182 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:12:00 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: [11/Sep/2024:08:12:00] ENGINE Bus STARTING 2024-09-11T08:12:01.182 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:12:00 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: CherryPy Checker: 2024-09-11T08:12:01.182 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:12:00 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: The Application mounted at '' has an empty config. 2024-09-11T08:12:01.182 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:12:00 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: 2024-09-11T08:12:01.182 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:12:00 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: [11/Sep/2024:08:12:00] ENGINE Serving on http://:::9283 2024-09-11T08:12:01.183 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:12:00 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: [11/Sep/2024:08:12:00] ENGINE Bus STARTED 2024-09-11T08:12:01.329 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:12:01.047+0000 7f81c1616ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:12:01.329 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:12:01] ENGINE Bus STARTING 2024-09-11T08:12:01.330 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: CherryPy Checker: 2024-09-11T08:12:01.330 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: The Application mounted at '' has an empty config. 2024-09-11T08:12:01.330 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:12:01.330 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:12:01] ENGINE Serving on http://:::9283 2024-09-11T08:12:01.330 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:12:01] ENGINE Bus STARTED 2024-09-11T08:12:02.001 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:12:02.001 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":69,"fsid":"566367e6-7014-11ef-bcea-c7b262605968","created":"2024-09-11T08:06:08.948724+0000","modified":"2024-09-11T08:12:01.049772+0000","last_up_change":"2024-09-11T08:11:12.015215+0000","last_in_change":"2024-09-11T08:10:54.456455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:09:13.770764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-11T08:11:23.512706+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-11T08:11:23.745619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T08:11:25.380463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T08:11:27.335637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T08:11:29.401048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"b831aea3-cb55-49cc-ad5c-3c340ec2f249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1906336816},{"type":"v1","addr":"172.21.15.71:6803","nonce":1906336816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1906336816},{"type":"v1","addr":"172.21.15.71:6805","nonce":1906336816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1906336816},{"type":"v1","addr":"172.21.15.71:6809","nonce":1906336816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1906336816},{"type":"v1","addr":"172.21.15.71:6807","nonce":1906336816}]},"public_addr":"172.21.15.71:6803/1906336816","cluster_addr":"172.21.15.71:6805/1906336816","heartbeat_back_addr":"172.21.15.71:6809/1906336816","heartbeat_front_addr":"172.21.15.71:6807/1906336816","state":["exists","up"]},{"osd":1,"uuid":"168eac60-ee21-42ca-85ea-844e9073441d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1800678301},{"type":"v1","addr":"172.21.15.71:6811","nonce":1800678301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1800678301},{"type":"v1","addr":"172.21.15.71:6813","nonce":1800678301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1800678301},{"type":"v1","addr":"172.21.15.71:6817","nonce":1800678301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1800678301},{"type":"v1","addr":"172.21.15.71:6815","nonce":1800678301}]},"public_addr":"172.21.15.71:6811/1800678301","cluster_addr":"172.21.15.71:6813/1800678301","heartbeat_back_addr":"172.21.15.71:6817/1800678301","heartbeat_front_addr":"172.21.15.71:6815/1800678301","state":["exists","up"]},{"osd":2,"uuid":"4c3bbd0e-1a4a-42ad-8cd8-29c663141966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1556215465},{"type":"v1","addr":"172.21.15.71:6819","nonce":1556215465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1556215465},{"type":"v1","addr":"172.21.15.71:6821","nonce":1556215465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1556215465},{"type":"v1","addr":"172.21.15.71:6825","nonce":1556215465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1556215465},{"type":"v1","addr":"172.21.15.71:6823","nonce":1556215465}]},"public_addr":"172.21.15.71:6819/1556215465","cluster_addr":"172.21.15.71:6821/1556215465","heartbeat_back_addr":"172.21.15.71:6825/1556215465","heartbeat_front_addr":"172.21.15.71:6823/1556215465","state":["exists","up"]},{"osd":3,"uuid":"418fdf0b-3c97-4e70-9696-0c879202d3b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2205658375},{"type":"v1","addr":"172.21.15.71:6827","nonce":2205658375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2205658375},{"type":"v1","addr":"172.21.15.71:6829","nonce":2205658375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2205658375},{"type":"v1","addr":"172.21.15.71:6833","nonce":2205658375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2205658375},{"type":"v1","addr":"172.21.15.71:6831","nonce":2205658375}]},"public_addr":"172.21.15.71:6827/2205658375","cluster_addr":"172.21.15.71:6829/2205658375","heartbeat_back_addr":"172.21.15.71:6833/2205658375","heartbeat_front_addr":"172.21.15.71:6831/2205658375","state":["exists","up"]},{"osd":4,"uuid":"f35baaaa-6a0f-45ee-8946-1c4ad32bbd4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2383004810},{"type":"v1","addr":"172.21.15.196:6801","nonce":2383004810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2383004810},{"type":"v1","addr":"172.21.15.196:6803","nonce":2383004810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2383004810},{"type":"v1","addr":"172.21.15.196:6807","nonce":2383004810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2383004810},{"type":"v1","addr":"172.21.15.196:6805","nonce":2383004810}]},"public_addr":"172.21.15.196:6801/2383004810","cluster_addr":"172.21.15.196:6803/2383004810","heartbeat_back_addr":"172.21.15.196:6807/2383004810","heartbeat_front_addr":"172.21.15.196:6805/2383004810","state":["exists","up"]},{"osd":5,"uuid":"84bbc0da-e641-42f8-87eb-68da7f4dbd5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3314011112},{"type":"v1","addr":"172.21.15.196:6809","nonce":3314011112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3314011112},{"type":"v1","addr":"172.21.15.196:6811","nonce":3314011112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3314011112},{"type":"v1","addr":"172.21.15.196:6815","nonce":3314011112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3314011112},{"type":"v1","addr":"172.21.15.196:6813","nonce":3314011112}]},"public_addr":"172.21.15.196:6809/3314011112","cluster_addr":"172.21.15.196:6811/3314011112","heartbeat_back_addr":"172.21.15.196:6815/3314011112","heartbeat_front_addr":"172.21.15.196:6813/3314011112","state":["exists","up"]},{"osd":6,"uuid":"cff32ed5-909b-4fac-9d6d-1ea4ef27d4b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4073919265},{"type":"v1","addr":"172.21.15.196:6817","nonce":4073919265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4073919265},{"type":"v1","addr":"172.21.15.196:6819","nonce":4073919265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4073919265},{"type":"v1","addr":"172.21.15.196:6823","nonce":4073919265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4073919265},{"type":"v1","addr":"172.21.15.196:6821","nonce":4073919265}]},"public_addr":"172.21.15.196:6817/4073919265","cluster_addr":"172.21.15.196:6819/4073919265","heartbeat_back_addr":"172.21.15.196:6823/4073919265","heartbeat_front_addr":"172.21.15.196:6821/4073919265","state":["exists","up"]},{"osd":7,"uuid":"1871ecef-9cb6-4705-9c26-032088cf97c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3168217828},{"type":"v1","addr":"172.21.15.196:6825","nonce":3168217828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3168217828},{"type":"v1","addr":"172.21.15.196:6827","nonce":3168217828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3168217828},{"type":"v1","addr":"172.21.15.196:6831","nonce":3168217828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3168217828},{"type":"v1","addr":"172.21.15.196:6829","nonce":3168217828}]},"public_addr":"172.21.15.196:6825/3168217828","cluster_addr":"172.21.15.196:6827/3168217828","heartbeat_back_addr":"172.21.15.196:6831/3168217828","heartbeat_front_addr":"172.21.15.196:6829/3168217828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:08:22.552887+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:08:47.326324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:09:11.931006+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:09:36.627722+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:09:58.222107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:10:22.165516+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:10:46.332026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:11:10.482672+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1292894177":"2024-09-12T08:12:01.049739+0000","172.21.15.71:0/2259311326":"2024-09-12T08:12:01.049739+0000","172.21.15.71:0/1243556948":"2024-09-12T08:06:29.435927+0000","172.21.15.71:0/1374934876":"2024-09-12T08:06:29.435927+0000","172.21.15.71:6800/1255934893":"2024-09-12T08:06:29.435927+0000","172.21.15.71:0/2106434609":"2024-09-12T08:12:01.049739+0000","172.21.15.71:6801/1255934893":"2024-09-12T08:06:29.435927+0000","172.21.15.71:0/2294552436":"2024-09-12T08:12:01.049739+0000","172.21.15.71:6801/3468024022":"2024-09-12T08:06:59.469628+0000","172.21.15.71:0/2967645637":"2024-09-12T08:06:29.435927+0000","172.21.15.71:0/1612684168":"2024-09-12T08:06:59.469628+0000","172.21.15.71:0/3643762338":"2024-09-12T08:06:59.469628+0000","172.21.15.71:0/799944144":"2024-09-12T08:12:01.049739+0000","172.21.15.71:6800/1121251349":"2024-09-12T08:12:01.049739+0000","172.21.15.71:6800/3468024022":"2024-09-12T08:06:59.469628+0000","172.21.15.71:0/3719406390":"2024-09-12T08:06:59.469628+0000","172.21.15.71:6801/1121251349":"2024-09-12T08:12:01.049739+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:12:02.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:12:02 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug there is no tcmu-runner data available 2024-09-11T08:12:02.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-11T08:12:02.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: Active manager daemon y restarted 2024-09-11T08:12:02.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: Activating manager daemon y 2024-09-11T08:12:02.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T08:12:02.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: mgrmap e19: y(active, starting, since 0.00755318s), standbys: x 2024-09-11T08:12:02.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:12:02.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:12:02.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:12:02.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:12:02.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:12:02.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:12:02.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:12:02.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:12:02.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:12:02.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:12:02.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:12:02.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:12:02.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:12:02.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:12:02.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:12:02.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:12:02.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: Manager daemon y is now available 2024-09-11T08:12:02.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:12:02.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:02.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:02.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:12:02.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:12:02.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:12:02.183 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:01 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:12:02.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-11T08:12:02.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: Active manager daemon y restarted 2024-09-11T08:12:02.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: Activating manager daemon y 2024-09-11T08:12:02.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T08:12:02.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: mgrmap e19: y(active, starting, since 0.00755318s), standbys: x 2024-09-11T08:12:02.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:12:02.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:12:02.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:12:02.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:12:02.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:12:02.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:12:02.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:12:02.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:12:02.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:12:02.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:12:02.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:12:02.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:12:02.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:12:02.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:12:02.196 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:12:02.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:12:02.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: Manager daemon y is now available 2024-09-11T08:12:02.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:12:02.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:02.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:02.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:12:02.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:12:02.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:12:02.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:12:02.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-11T08:12:02.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: Active manager daemon y restarted 2024-09-11T08:12:02.198 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: Activating manager daemon y 2024-09-11T08:12:02.198 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T08:12:02.198 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: mgrmap e19: y(active, starting, since 0.00755318s), standbys: x 2024-09-11T08:12:02.198 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:12:02.198 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:12:02.198 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:12:02.198 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:12:02.198 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:12:02.198 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:12:02.198 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:12:02.198 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:12:02.198 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:12:02.198 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:12:02.198 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:12:02.199 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:12:02.199 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:12:02.199 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:12:02.199 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:12:02.199 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:12:02.199 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: Manager daemon y is now available 2024-09-11T08:12:02.199 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:12:02.199 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:02.199 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:02.199 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:12:02.199 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:12:02.199 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:12:02.199 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:01 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:12:02.200 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:12:02.134+0000 7f8164398640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:12:02] ENGINE Error in HTTPServer.serve 2024-09-11T08:12:02.200 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: Traceback (most recent call last): 2024-09-11T08:12:02.200 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:12:02.200 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: self._connections.run(self.expiration_interval) 2024-09-11T08:12:02.200 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:12:02.200 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: self._run(expiration_interval) 2024-09-11T08:12:02.200 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:12:02.200 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:12:02.200 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:12:02.200 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:12:02.200 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:12:02.200 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: s = self.context.wrap_socket( 2024-09-11T08:12:02.200 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:12:02.201 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: return self.sslsocket_class._create( 2024-09-11T08:12:02.201 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:12:02.201 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: self.do_handshake() 2024-09-11T08:12:02.201 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:12:02.201 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: self._sslobj.do_handshake() 2024-09-11T08:12:02.201 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:12:02.201 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:12:03.028 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T08:12:03.028 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:12:03.179 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:02 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/2320521931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:12:03.179 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:02 smithi196 ceph-mon[30677]: [11/Sep/2024:08:12:02] ENGINE Bus STARTING 2024-09-11T08:12:03.179 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:02 smithi196 ceph-mon[30677]: mgrmap e20: y(active, since 1.01152s), standbys: x 2024-09-11T08:12:03.179 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:02 smithi196 ceph-mon[30677]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:03.179 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:02 smithi196 ceph-mon[30677]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:03.179 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:02 smithi196 ceph-mon[30677]: [11/Sep/2024:08:12:02] ENGINE Serving on https://172.21.15.71:7150 2024-09-11T08:12:03.179 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:02 smithi196 ceph-mon[30677]: [11/Sep/2024:08:12:02] ENGINE Bus STARTED 2024-09-11T08:12:03.179 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:02 smithi196 ceph-mon[30677]: [11/Sep/2024:08:12:02] ENGINE Error in HTTPServer.serve 2024-09-11T08:12:03.179 INFO:journalctl@ceph.mon.b.smithi196.stdout: Traceback (most recent call last): 2024-09-11T08:12:03.179 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:12:03.179 INFO:journalctl@ceph.mon.b.smithi196.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:12:03.180 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:12:03.180 INFO:journalctl@ceph.mon.b.smithi196.stdout: self._run(expiration_interval) 2024-09-11T08:12:03.180 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:12:03.180 INFO:journalctl@ceph.mon.b.smithi196.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:12:03.180 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:12:03.180 INFO:journalctl@ceph.mon.b.smithi196.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:12:03.180 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:12:03.180 INFO:journalctl@ceph.mon.b.smithi196.stdout: s = self.context.wrap_socket( 2024-09-11T08:12:03.180 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:12:03.180 INFO:journalctl@ceph.mon.b.smithi196.stdout: return self.sslsocket_class._create( 2024-09-11T08:12:03.180 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:12:03.180 INFO:journalctl@ceph.mon.b.smithi196.stdout: self.do_handshake() 2024-09-11T08:12:03.180 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:12:03.181 INFO:journalctl@ceph.mon.b.smithi196.stdout: self._sslobj.do_handshake() 2024-09-11T08:12:03.181 INFO:journalctl@ceph.mon.b.smithi196.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:12:03.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:02 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:03.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2320521931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:12:03.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[25834]: [11/Sep/2024:08:12:02] ENGINE Bus STARTING 2024-09-11T08:12:03.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[25834]: mgrmap e20: y(active, since 1.01152s), standbys: x 2024-09-11T08:12:03.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[25834]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:03.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[25834]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:03.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[25834]: [11/Sep/2024:08:12:02] ENGINE Serving on https://172.21.15.71:7150 2024-09-11T08:12:03.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[25834]: [11/Sep/2024:08:12:02] ENGINE Bus STARTED 2024-09-11T08:12:03.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[25834]: [11/Sep/2024:08:12:02] ENGINE Error in HTTPServer.serve 2024-09-11T08:12:03.196 INFO:journalctl@ceph.mon.a.smithi071.stdout: Traceback (most recent call last): 2024-09-11T08:12:03.196 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:12:03.196 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:12:03.196 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:12:03.196 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._run(expiration_interval) 2024-09-11T08:12:03.196 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:12:03.196 INFO:journalctl@ceph.mon.a.smithi071.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:12:03.196 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:12:03.196 INFO:journalctl@ceph.mon.a.smithi071.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:12:03.196 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:12:03.196 INFO:journalctl@ceph.mon.a.smithi071.stdout: s = self.context.wrap_socket( 2024-09-11T08:12:03.197 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:12:03.197 INFO:journalctl@ceph.mon.a.smithi071.stdout: return self.sslsocket_class._create( 2024-09-11T08:12:03.197 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:12:03.197 INFO:journalctl@ceph.mon.a.smithi071.stdout: self.do_handshake() 2024-09-11T08:12:03.197 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:12:03.197 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._sslobj.do_handshake() 2024-09-11T08:12:03.197 INFO:journalctl@ceph.mon.a.smithi071.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:12:03.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:03.197 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/2320521931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:12:03.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[39897]: [11/Sep/2024:08:12:02] ENGINE Bus STARTING 2024-09-11T08:12:03.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[39897]: mgrmap e20: y(active, since 1.01152s), standbys: x 2024-09-11T08:12:03.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[39897]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:03.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[39897]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:03.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[39897]: [11/Sep/2024:08:12:02] ENGINE Serving on https://172.21.15.71:7150 2024-09-11T08:12:03.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[39897]: [11/Sep/2024:08:12:02] ENGINE Bus STARTED 2024-09-11T08:12:03.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[39897]: [11/Sep/2024:08:12:02] ENGINE Error in HTTPServer.serve 2024-09-11T08:12:03.198 INFO:journalctl@ceph.mon.c.smithi071.stdout: Traceback (most recent call last): 2024-09-11T08:12:03.198 INFO:journalctl@ceph.mon.c.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:12:03.198 INFO:journalctl@ceph.mon.c.smithi071.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:12:03.198 INFO:journalctl@ceph.mon.c.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:12:03.198 INFO:journalctl@ceph.mon.c.smithi071.stdout: self._run(expiration_interval) 2024-09-11T08:12:03.198 INFO:journalctl@ceph.mon.c.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:12:03.199 INFO:journalctl@ceph.mon.c.smithi071.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:12:03.199 INFO:journalctl@ceph.mon.c.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:12:03.199 INFO:journalctl@ceph.mon.c.smithi071.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:12:03.199 INFO:journalctl@ceph.mon.c.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:12:03.199 INFO:journalctl@ceph.mon.c.smithi071.stdout: s = self.context.wrap_socket( 2024-09-11T08:12:03.199 INFO:journalctl@ceph.mon.c.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:12:03.199 INFO:journalctl@ceph.mon.c.smithi071.stdout: return self.sslsocket_class._create( 2024-09-11T08:12:03.199 INFO:journalctl@ceph.mon.c.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:12:03.199 INFO:journalctl@ceph.mon.c.smithi071.stdout: self.do_handshake() 2024-09-11T08:12:03.199 INFO:journalctl@ceph.mon.c.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:12:03.199 INFO:journalctl@ceph.mon.c.smithi071.stdout: self._sslobj.do_handshake() 2024-09-11T08:12:03.199 INFO:journalctl@ceph.mon.c.smithi071.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:12:03.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:02 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:03.458 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:04.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:04 smithi071 ceph-mon[39897]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:04.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:04 smithi071 ceph-mon[39897]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-11T08:12:04.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:04 smithi071 ceph-mon[25834]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:04.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:04 smithi071 ceph-mon[25834]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-11T08:12:04.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:04 smithi196 ceph-mon[30677]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:04.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:04 smithi196 ceph-mon[30677]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-11T08:12:05.243 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.243 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.243 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.243 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.243 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.243 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.243 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.244 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.244 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.244 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.244 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.244 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.244 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.244 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.244 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.244 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.244 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.244 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.245 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.245 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.245 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.245 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.245 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.245 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.245 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.245 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.245 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.245 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.245 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.245 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:05 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:05 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:05 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:05 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:05 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:05 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:05 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:05 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:05 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:05 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:05 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:05 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:05 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:05.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:05 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:05 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:05 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:05.492 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:12:05.492 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":69,"fsid":"566367e6-7014-11ef-bcea-c7b262605968","created":"2024-09-11T08:06:08.948724+0000","modified":"2024-09-11T08:12:01.049772+0000","last_up_change":"2024-09-11T08:11:12.015215+0000","last_in_change":"2024-09-11T08:10:54.456455+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:09:13.770764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-11T08:11:23.512706+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-11T08:11:23.745619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T08:11:25.380463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T08:11:27.335637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T08:11:29.401048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"b831aea3-cb55-49cc-ad5c-3c340ec2f249","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1906336816},{"type":"v1","addr":"172.21.15.71:6803","nonce":1906336816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1906336816},{"type":"v1","addr":"172.21.15.71:6805","nonce":1906336816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1906336816},{"type":"v1","addr":"172.21.15.71:6809","nonce":1906336816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1906336816},{"type":"v1","addr":"172.21.15.71:6807","nonce":1906336816}]},"public_addr":"172.21.15.71:6803/1906336816","cluster_addr":"172.21.15.71:6805/1906336816","heartbeat_back_addr":"172.21.15.71:6809/1906336816","heartbeat_front_addr":"172.21.15.71:6807/1906336816","state":["exists","up"]},{"osd":1,"uuid":"168eac60-ee21-42ca-85ea-844e9073441d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1800678301},{"type":"v1","addr":"172.21.15.71:6811","nonce":1800678301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1800678301},{"type":"v1","addr":"172.21.15.71:6813","nonce":1800678301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1800678301},{"type":"v1","addr":"172.21.15.71:6817","nonce":1800678301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1800678301},{"type":"v1","addr":"172.21.15.71:6815","nonce":1800678301}]},"public_addr":"172.21.15.71:6811/1800678301","cluster_addr":"172.21.15.71:6813/1800678301","heartbeat_back_addr":"172.21.15.71:6817/1800678301","heartbeat_front_addr":"172.21.15.71:6815/1800678301","state":["exists","up"]},{"osd":2,"uuid":"4c3bbd0e-1a4a-42ad-8cd8-29c663141966","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1556215465},{"type":"v1","addr":"172.21.15.71:6819","nonce":1556215465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1556215465},{"type":"v1","addr":"172.21.15.71:6821","nonce":1556215465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1556215465},{"type":"v1","addr":"172.21.15.71:6825","nonce":1556215465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1556215465},{"type":"v1","addr":"172.21.15.71:6823","nonce":1556215465}]},"public_addr":"172.21.15.71:6819/1556215465","cluster_addr":"172.21.15.71:6821/1556215465","heartbeat_back_addr":"172.21.15.71:6825/1556215465","heartbeat_front_addr":"172.21.15.71:6823/1556215465","state":["exists","up"]},{"osd":3,"uuid":"418fdf0b-3c97-4e70-9696-0c879202d3b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2205658375},{"type":"v1","addr":"172.21.15.71:6827","nonce":2205658375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2205658375},{"type":"v1","addr":"172.21.15.71:6829","nonce":2205658375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2205658375},{"type":"v1","addr":"172.21.15.71:6833","nonce":2205658375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2205658375},{"type":"v1","addr":"172.21.15.71:6831","nonce":2205658375}]},"public_addr":"172.21.15.71:6827/2205658375","cluster_addr":"172.21.15.71:6829/2205658375","heartbeat_back_addr":"172.21.15.71:6833/2205658375","heartbeat_front_addr":"172.21.15.71:6831/2205658375","state":["exists","up"]},{"osd":4,"uuid":"f35baaaa-6a0f-45ee-8946-1c4ad32bbd4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2383004810},{"type":"v1","addr":"172.21.15.196:6801","nonce":2383004810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2383004810},{"type":"v1","addr":"172.21.15.196:6803","nonce":2383004810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2383004810},{"type":"v1","addr":"172.21.15.196:6807","nonce":2383004810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2383004810},{"type":"v1","addr":"172.21.15.196:6805","nonce":2383004810}]},"public_addr":"172.21.15.196:6801/2383004810","cluster_addr":"172.21.15.196:6803/2383004810","heartbeat_back_addr":"172.21.15.196:6807/2383004810","heartbeat_front_addr":"172.21.15.196:6805/2383004810","state":["exists","up"]},{"osd":5,"uuid":"84bbc0da-e641-42f8-87eb-68da7f4dbd5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3314011112},{"type":"v1","addr":"172.21.15.196:6809","nonce":3314011112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3314011112},{"type":"v1","addr":"172.21.15.196:6811","nonce":3314011112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3314011112},{"type":"v1","addr":"172.21.15.196:6815","nonce":3314011112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3314011112},{"type":"v1","addr":"172.21.15.196:6813","nonce":3314011112}]},"public_addr":"172.21.15.196:6809/3314011112","cluster_addr":"172.21.15.196:6811/3314011112","heartbeat_back_addr":"172.21.15.196:6815/3314011112","heartbeat_front_addr":"172.21.15.196:6813/3314011112","state":["exists","up"]},{"osd":6,"uuid":"cff32ed5-909b-4fac-9d6d-1ea4ef27d4b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4073919265},{"type":"v1","addr":"172.21.15.196:6817","nonce":4073919265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4073919265},{"type":"v1","addr":"172.21.15.196:6819","nonce":4073919265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4073919265},{"type":"v1","addr":"172.21.15.196:6823","nonce":4073919265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4073919265},{"type":"v1","addr":"172.21.15.196:6821","nonce":4073919265}]},"public_addr":"172.21.15.196:6817/4073919265","cluster_addr":"172.21.15.196:6819/4073919265","heartbeat_back_addr":"172.21.15.196:6823/4073919265","heartbeat_front_addr":"172.21.15.196:6821/4073919265","state":["exists","up"]},{"osd":7,"uuid":"1871ecef-9cb6-4705-9c26-032088cf97c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3168217828},{"type":"v1","addr":"172.21.15.196:6825","nonce":3168217828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3168217828},{"type":"v1","addr":"172.21.15.196:6827","nonce":3168217828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3168217828},{"type":"v1","addr":"172.21.15.196:6831","nonce":3168217828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3168217828},{"type":"v1","addr":"172.21.15.196:6829","nonce":3168217828}]},"public_addr":"172.21.15.196:6825/3168217828","cluster_addr":"172.21.15.196:6827/3168217828","heartbeat_back_addr":"172.21.15.196:6831/3168217828","heartbeat_front_addr":"172.21.15.196:6829/3168217828","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:08:22.552887+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:08:47.326324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:09:11.931006+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:09:36.627722+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:09:58.222107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:10:22.165516+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:10:46.332026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:11:10.482672+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1292894177":"2024-09-12T08:12:01.049739+0000","172.21.15.71:0/2259311326":"2024-09-12T08:12:01.049739+0000","172.21.15.71:0/1243556948":"2024-09-12T08:06:29.435927+0000","172.21.15.71:0/1374934876":"2024-09-12T08:06:29.435927+0000","172.21.15.71:6800/1255934893":"2024-09-12T08:06:29.435927+0000","172.21.15.71:0/2106434609":"2024-09-12T08:12:01.049739+0000","172.21.15.71:6801/1255934893":"2024-09-12T08:06:29.435927+0000","172.21.15.71:0/2294552436":"2024-09-12T08:12:01.049739+0000","172.21.15.71:6801/3468024022":"2024-09-12T08:06:59.469628+0000","172.21.15.71:0/2967645637":"2024-09-12T08:06:29.435927+0000","172.21.15.71:0/1612684168":"2024-09-12T08:06:59.469628+0000","172.21.15.71:0/3643762338":"2024-09-12T08:06:59.469628+0000","172.21.15.71:0/799944144":"2024-09-12T08:12:01.049739+0000","172.21.15.71:6800/1121251349":"2024-09-12T08:12:01.049739+0000","172.21.15.71:6800/3468024022":"2024-09-12T08:06:59.469628+0000","172.21.15.71:0/3719406390":"2024-09-12T08:06:59.469628+0000","172.21.15.71:6801/1121251349":"2024-09-12T08:12:01.049739+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:12:06.300 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T08:12:06.300 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T08:12:06.301 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T08:12:06.301 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T08:12:06.302 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T08:12:06.302 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T08:12:06.302 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T08:12:06.303 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T08:12:06.349 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: Adjusting osd_memory_target on smithi196 to 3760M 2024-09-11T08:12:06.349 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:06.349 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/3690499603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:12:06.349 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:06.349 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:06.349 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.350 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.350 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.350 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.350 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.350 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.350 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.350 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.350 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:06.350 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:12:06.350 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:12:06.350 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[39897]: mgrmap e22: y(active, since 5s), standbys: x 2024-09-11T08:12:06.351 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: Adjusting osd_memory_target on smithi196 to 3760M 2024-09-11T08:12:06.351 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:06.351 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3690499603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:12:06.351 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:06.351 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:06.351 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.351 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.351 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.352 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.352 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.352 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.352 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.352 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.352 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:06.353 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:12:06.353 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:12:06.353 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:06 smithi071 ceph-mon[25834]: mgrmap e22: y(active, since 5s), standbys: x 2024-09-11T08:12:06.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: Adjusting osd_memory_target on smithi196 to 3760M 2024-09-11T08:12:06.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:06.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/3690499603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:12:06.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:06.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:06.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:12:06.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:06.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:12:06.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:12:06.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-mon[30677]: mgrmap e22: y(active, since 5s), standbys: x 2024-09-11T08:12:06.769 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:06.851 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:06.851 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:06.931 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:12:06 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: ::ffff:172.21.15.196 - - [11/Sep/2024:08:12:06] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:07.187 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:07.187 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:07.187 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:07.187 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:07.188 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:07.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[39897]: Adjusting osd_memory_target on smithi071 to 3760M 2024-09-11T08:12:07.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[39897]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-11T08:12:07.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[39897]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-11T08:12:07.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[39897]: Updating smithi071:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:12:07.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[39897]: Updating smithi196:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:12:07.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[39897]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:12:07.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[39897]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:12:07.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[39897]: Updating smithi196:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:12:07.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:07.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:07.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:07.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:07.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:07.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[25834]: Adjusting osd_memory_target on smithi071 to 3760M 2024-09-11T08:12:07.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[25834]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-11T08:12:07.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[25834]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-11T08:12:07.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[25834]: Updating smithi071:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:12:07.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[25834]: Updating smithi196:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:12:07.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[25834]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:12:07.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[25834]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:12:07.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[25834]: Updating smithi196:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:12:07.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:07.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:07.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:07.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:07.197 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:07 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:07.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:07 smithi196 ceph-mon[30677]: Adjusting osd_memory_target on smithi071 to 3760M 2024-09-11T08:12:07.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:07 smithi196 ceph-mon[30677]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-11T08:12:07.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:07 smithi196 ceph-mon[30677]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-11T08:12:07.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:07 smithi196 ceph-mon[30677]: Updating smithi071:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:12:07.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:07 smithi196 ceph-mon[30677]: Updating smithi196:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:12:07.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:07 smithi196 ceph-mon[30677]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:12:07.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:07 smithi196 ceph-mon[30677]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:12:07.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:07 smithi196 ceph-mon[30677]: Updating smithi196:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:12:07.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:07 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:07.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:07 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:07.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:07 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:07.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:07 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:07.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:07 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:08.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:08 smithi071 ceph-mon[25834]: Updating smithi071:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:12:08.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:08 smithi071 ceph-mon[25834]: Deploying daemon alertmanager.a on smithi071 2024-09-11T08:12:08.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:08 smithi071 ceph-mon[25834]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T08:12:08.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:08 smithi071 ceph-mon[39897]: Updating smithi071:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:12:08.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:08 smithi071 ceph-mon[39897]: Deploying daemon alertmanager.a on smithi071 2024-09-11T08:12:08.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:08 smithi071 ceph-mon[39897]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T08:12:08.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:08 smithi196 ceph-mon[30677]: Updating smithi071:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:12:08.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:08 smithi196 ceph-mon[30677]: Deploying daemon alertmanager.a on smithi071 2024-09-11T08:12:08.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:08 smithi196 ceph-mon[30677]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T08:12:10.165 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:10 smithi071 ceph-mon[39897]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T08:12:10.165 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:10 smithi071 ceph-mon[25834]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T08:12:10.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:10 smithi196 ceph-mon[30677]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T08:12:11.495 INFO:teuthology.orchestra.run.smithi071.stdout:231928233998 2024-09-11T08:12:11.495 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T08:12:12.307 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:12 smithi071 ceph-mon[39897]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:12:12.307 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:12 smithi071 ceph-mon[25834]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:12:12.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:12 smithi196 ceph-mon[30677]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:12:12.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:12:12 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug there is no tcmu-runner data available 2024-09-11T08:12:13.321 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:13 smithi071 ceph-mon[39897]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:13.321 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:13 smithi071 ceph-mon[25834]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:13.420 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:13 smithi196 ceph-mon[30677]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:14.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:14 smithi071 ceph-mon[39897]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:12:14.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:14 smithi071 ceph-mon[25834]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:12:14.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:14 smithi196 ceph-mon[30677]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:12:16.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:16 smithi196 ceph-mon[30677]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:12:16.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:16 smithi071 ceph-mon[39897]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:12:16.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:16 smithi071 ceph-mon[25834]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:12:16.931 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:12:16 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: ::ffff:172.21.15.196 - - [11/Sep/2024:08:12:16] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:17.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:17 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:17.441 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:17 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:17.441 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:17 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:17.463 INFO:teuthology.orchestra.run.smithi071.stdout:107374182433 2024-09-11T08:12:17.464 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T08:12:18.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:18 smithi196 ceph-mon[30677]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:12:18.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:18 smithi071 ceph-mon[25834]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:12:18.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:18 smithi071 ceph-mon[39897]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:12:20.367 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:20 smithi071 ceph-mon[39897]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:20.367 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:20 smithi071 ceph-mon[25834]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:20.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:20 smithi196 ceph-mon[30677]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:21.390 INFO:teuthology.orchestra.run.smithi071.stdout:34359738418 2024-09-11T08:12:21.390 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T08:12:22.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:12:22 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug there is no tcmu-runner data available 2024-09-11T08:12:22.435 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:22 smithi196 ceph-mon[30677]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:22.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:22 smithi071 ceph-mon[25834]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:22.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:22 smithi071 ceph-mon[39897]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:23.386 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:23 smithi071 ceph-mon[39897]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:23.386 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:23 smithi071 ceph-mon[25834]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:23.409 INFO:teuthology.orchestra.run.smithi071.stdout:55834574894 2024-09-11T08:12:23.410 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T08:12:23.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:23 smithi196 ceph-mon[30677]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:24.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:24 smithi196 ceph-mon[30677]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:12:24.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:24 smithi071 ceph-mon[39897]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:12:24.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:24 smithi071 ceph-mon[25834]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:12:26.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:26 smithi196 ceph-mon[30677]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:26.444 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:26 smithi071 ceph-mon[39897]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:26.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:26 smithi071 ceph-mon[25834]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:26.931 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:12:26 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: ::ffff:172.21.15.196 - - [11/Sep/2024:08:12:26] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:27.575 INFO:teuthology.orchestra.run.smithi071.stdout:77309411369 2024-09-11T08:12:27.575 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T08:12:28.382 INFO:teuthology.orchestra.run.smithi071.stdout:128849018911 2024-09-11T08:12:28.383 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T08:12:28.409 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:28 smithi071 ceph-mon[39897]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:28.409 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:28 smithi071 ceph-mon[25834]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:28.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:28 smithi196 ceph-mon[30677]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:28.664 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:28.667 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:29.058 INFO:teuthology.orchestra.run.smithi071.stdout:197568495638 2024-09-11T08:12:29.058 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T08:12:29.495 INFO:teuthology.orchestra.run.smithi071.stdout:163208757274 2024-09-11T08:12:29.495 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T08:12:29.521 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:29.540 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:29.547 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:30.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:30 smithi071 ceph-mon[25834]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:30.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:30 smithi071 ceph-mon[39897]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:30.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:30 smithi196 ceph-mon[30677]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:31.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:31 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:31.442 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:31 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:31.442 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:12:31 smithi071 systemd[1]: Started Ceph alertmanager.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:12:31.442 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:31 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:31.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:12:31] ENGINE Bus STOPPING 2024-09-11T08:12:32.195 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:12:31] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:12:32.195 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:12:31] ENGINE Bus STOPPED 2024-09-11T08:12:32.195 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:12:31] ENGINE Bus STARTING 2024-09-11T08:12:32.195 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:12:31] ENGINE Serving on http://:::9283 2024-09-11T08:12:32.195 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:12:31 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:12:31] ENGINE Bus STARTED 2024-09-11T08:12:32.382 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:12:32 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug there is no tcmu-runner data available 2024-09-11T08:12:32.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:32 smithi196 ceph-mon[30677]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:32.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:32 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:32 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:32 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:32 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:32 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:32 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:32 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:12:32.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:32 smithi196 ceph-mon[30677]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:12:32.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:32 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:32 smithi196 ceph-mon[30677]: Deploying daemon grafana.a on smithi196 2024-09-11T08:12:32.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[39897]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:32.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:12:32.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[39897]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:12:32.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[39897]: Deploying daemon grafana.a on smithi196 2024-09-11T08:12:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[25834]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:12:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[25834]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:12:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:32.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:32 smithi071 ceph-mon[25834]: Deploying daemon grafana.a on smithi196 2024-09-11T08:12:32.885 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:32.886 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:33.328 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:12:33 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a[83304]: ts=2024-09-11T08:12:33.228Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000625558s 2024-09-11T08:12:33.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:33 smithi196 ceph-mon[30677]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:33.694 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:33 smithi071 ceph-mon[25834]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:33.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:33 smithi071 ceph-mon[39897]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:34.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:34 smithi196 ceph-mon[30677]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:34.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:34 smithi071 ceph-mon[25834]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:34.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:34 smithi071 ceph-mon[39897]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:34.965 INFO:teuthology.orchestra.run.smithi071.stdout:107374182438 2024-09-11T08:12:35.612 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:35 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3881498261' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:12:35.612 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:35 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/3881498261' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:12:35.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:35 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/3881498261' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:12:36.584 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:36 smithi071 ceph-mon[39897]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:36.584 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:36 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:36.584 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:36 smithi071 ceph-mon[25834]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:36.584 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:36 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:36.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:36 smithi196 ceph-mon[30677]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:36.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:36 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:36.681 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:12:36 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: ::ffff:172.21.15.196 - - [11/Sep/2024:08:12:36] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:36.901 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182433 got 107374182438 for osd.3 2024-09-11T08:12:36.902 DEBUG:teuthology.parallel:result is None 2024-09-11T08:12:38.647 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:38 smithi071 ceph-mon[39897]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:38.647 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:38 smithi071 ceph-mon[25834]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:38.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:38 smithi196 ceph-mon[30677]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:39.186 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:39.337 INFO:teuthology.orchestra.run.smithi071.stdout:128849018914 2024-09-11T08:12:39.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:39 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/45953798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:12:39.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:39 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/45953798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:12:39.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:39 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/45953798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:12:40.050 INFO:teuthology.orchestra.run.smithi071.stdout:197568495641 2024-09-11T08:12:40.396 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018911 got 128849018914 for osd.4 2024-09-11T08:12:40.396 DEBUG:teuthology.parallel:result is None 2024-09-11T08:12:40.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:40 smithi196 ceph-mon[30677]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:40 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/2449140616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:12:40.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:40 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/975724541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:12:40.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:40 smithi071 ceph-mon[39897]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:40.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:40 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/2449140616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:12:40.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:40 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/975724541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:12:40.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:40 smithi071 ceph-mon[25834]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:40.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:40 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2449140616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:12:40.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:40 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/975724541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:12:41.016 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495638 got 197568495641 for osd.6 2024-09-11T08:12:41.017 DEBUG:teuthology.parallel:result is None 2024-09-11T08:12:41.312 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:12:41 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a[83304]: ts=2024-09-11T08:12:41.230Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003243389s 2024-09-11T08:12:41.365 INFO:teuthology.orchestra.run.smithi071.stdout:55834574897 2024-09-11T08:12:41.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:41 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/2165546625' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:12:41.663 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:41 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/2165546625' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:12:41.663 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:41 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2165546625' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:12:42.412 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:12:42 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug there is no tcmu-runner data available 2024-09-11T08:12:42.611 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:42 smithi071 ceph-mon[39897]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:42.612 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:42 smithi071 ceph-mon[25834]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:42.635 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574894 got 55834574897 for osd.1 2024-09-11T08:12:42.635 DEBUG:teuthology.parallel:result is None 2024-09-11T08:12:42.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:42 smithi196 ceph-mon[30677]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:42.753 INFO:teuthology.orchestra.run.smithi071.stdout:77309411372 2024-09-11T08:12:42.885 INFO:teuthology.orchestra.run.smithi071.stdout:231928234004 2024-09-11T08:12:43.483 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:43 smithi071 ceph-mon[39897]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:43.483 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:43 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/685767071' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:12:43.484 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:43 smithi071 ceph-mon[25834]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:43.484 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:43 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/685767071' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:12:43.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-mon[30677]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:43.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/685767071' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:12:43.682 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 podman[72370]: 2024-09-11 08:12:43.626664105 +0000 UTC m=+0.369179182 container start b20068dfb13a99b8377e0f18493035f8c7fa2c604cb9db11e3955cb4940865da (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, version=8.5, vendor=Red Hat, Inc., maintainer=Paul Cuzner , description=Ceph Grafana Container, io.k8s.display-name=Red Hat Universal Base Image 8, release=239.1651231664, vcs-type=git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=ubi8-container, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-04-29T11:48:11.055897, architecture=x86_64, io.openshift.tags=base rhel8, io.openshift.expose-services=, distribution-scope=public, name=ubi8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56) 2024-09-11T08:12:43.976 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 bash[72370]: b20068dfb13a99b8377e0f18493035f8c7fa2c604cb9db11e3955cb4940865da 2024-09-11T08:12:43.976 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 systemd[1]: Started Ceph grafana.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:12:43.976 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=settings t=2024-09-11T08:12:43.792594872Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-11T08:12:43.976 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=settings t=2024-09-11T08:12:43.792783579Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-11T08:12:43.976 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=settings t=2024-09-11T08:12:43.792802742Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-11T08:12:43.977 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=settings t=2024-09-11T08:12:43.792810078Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-11T08:12:43.977 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=settings t=2024-09-11T08:12:43.792816681Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-11T08:12:43.977 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=settings t=2024-09-11T08:12:43.792822842Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-11T08:12:43.977 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=settings t=2024-09-11T08:12:43.792828543Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-11T08:12:43.977 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=settings t=2024-09-11T08:12:43.792834377Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-11T08:12:43.977 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=settings t=2024-09-11T08:12:43.792840191Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-11T08:12:43.977 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=settings t=2024-09-11T08:12:43.792845917Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-11T08:12:43.977 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=settings t=2024-09-11T08:12:43.792851503Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-11T08:12:43.977 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=settings t=2024-09-11T08:12:43.792856851Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-11T08:12:43.978 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=settings t=2024-09-11T08:12:43.792862442Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-11T08:12:43.978 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=settings t=2024-09-11T08:12:43.792868746Z level=info msg="App mode production" 2024-09-11T08:12:43.978 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=sqlstore t=2024-09-11T08:12:43.792931008Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-11T08:12:43.978 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=sqlstore t=2024-09-11T08:12:43.792948327Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-11T08:12:43.978 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.793788729Z level=info msg="Starting DB migrations" 2024-09-11T08:12:43.978 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.793865352Z level=info msg="Executing migration" id="create migration_log table" 2024-09-11T08:12:43.978 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.794467646Z level=info msg="Executing migration" id="create user table" 2024-09-11T08:12:43.978 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.7949043Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-11T08:12:43.978 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.795697539Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-11T08:12:43.978 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.796348851Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-11T08:12:43.978 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.797111249Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-11T08:12:43.978 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.797971902Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-11T08:12:43.979 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.798707007Z level=info msg="Executing migration" id="create user table v2" 2024-09-11T08:12:43.979 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.79956857Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-11T08:12:43.979 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.800355753Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-11T08:12:43.979 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.800850853Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-11T08:12:43.979 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.801645641Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-11T08:12:43.979 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.80251112Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-11T08:12:43.979 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.802965912Z level=info msg="Executing migration" id="Update user table charset" 2024-09-11T08:12:43.979 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.803811991Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-11T08:12:43.979 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.804670764Z level=info msg="Executing migration" id="Add missing user data" 2024-09-11T08:12:43.979 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.805519186Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-11T08:12:43.979 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.80631465Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-11T08:12:43.979 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.807144466Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-11T08:12:43.979 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.807969566Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-11T08:12:43.981 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.809187571Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-11T08:12:43.981 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.810311649Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-11T08:12:43.981 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.810798276Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-11T08:12:43.981 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.811238727Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-11T08:12:43.981 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.811631546Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-11T08:12:43.981 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.812058305Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-11T08:12:43.981 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.812355495Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-11T08:12:43.981 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.812710513Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-11T08:12:43.981 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.813114659Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-11T08:12:43.982 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.813504544Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-11T08:12:43.982 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.813887867Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-11T08:12:43.982 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.814532421Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-11T08:12:43.982 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.814942788Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-11T08:12:43.982 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.815405614Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-11T08:12:43.982 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.815757608Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-11T08:12:43.982 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.816181461Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-11T08:12:43.982 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.816654402Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-11T08:12:43.982 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.81706937Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-11T08:12:43.982 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.817442694Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-11T08:12:43.982 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.817771436Z level=info msg="Executing migration" id="create star table" 2024-09-11T08:12:43.982 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.818168937Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-11T08:12:43.982 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.818587869Z level=info msg="Executing migration" id="create org table v1" 2024-09-11T08:12:43.982 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.818969921Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-11T08:12:43.983 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.819362822Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-11T08:12:43.983 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.819760163Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-11T08:12:43.983 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.820181547Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-11T08:12:43.983 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.820589229Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-11T08:12:43.983 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.82099483Z level=info msg="Executing migration" id="Update org table charset" 2024-09-11T08:12:43.983 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.821307587Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-11T08:12:43.983 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.822061844Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-11T08:12:43.983 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.822382291Z level=info msg="Executing migration" id="create dashboard table" 2024-09-11T08:12:43.983 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.822920547Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-11T08:12:43.983 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.82332165Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-11T08:12:43.983 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.824171981Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-11T08:12:43.983 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.82498922Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-11T08:12:43.983 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.825833266Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-11T08:12:43.983 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.826232343Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-11T08:12:43.984 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.827074741Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-11T08:12:43.984 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.82786991Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-11T08:12:43.984 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.828748958Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-11T08:12:43.984 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.829182403Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-11T08:12:43.984 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.830011731Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-11T08:12:43.984 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.830849378Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-11T08:12:43.984 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.831687569Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-11T08:12:43.984 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.832171909Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-11T08:12:43.984 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.83301649Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-11T08:12:43.984 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.833895493Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-11T08:12:43.984 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.83473615Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-11T08:12:43.984 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.835221778Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-11T08:12:43.984 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.83603552Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-11T08:12:43.985 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.836881179Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-11T08:12:43.985 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.837717267Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-11T08:12:43.985 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.837994942Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-11T08:12:43.985 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.838573979Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-11T08:12:43.985 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.839064767Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-11T08:12:43.985 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.839908002Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-11T08:12:43.985 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.840774565Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-11T08:12:43.985 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.841100218Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-11T08:12:43.985 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.841988149Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-11T08:12:43.985 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.842855027Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-11T08:12:43.985 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.843677212Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-11T08:12:43.985 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.84409652Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-11T08:12:43.985 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.846956024Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-11T08:12:43.985 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.849848557Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-11T08:12:43.986 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.852559687Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-11T08:12:43.986 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.852972429Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-11T08:12:43.986 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.855821292Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-11T08:12:43.986 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.858749679Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-11T08:12:43.986 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.861482204Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-11T08:12:43.986 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.861980071Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-11T08:12:43.986 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.86481826Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-11T08:12:43.986 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.867618619Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-11T08:12:43.986 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.870366088Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-11T08:12:43.986 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.870795686Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-11T08:12:43.986 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.873416038Z level=info msg="Executing migration" id="create data_source table" 2024-09-11T08:12:43.986 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.876151032Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-11T08:12:43.986 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.876594895Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-11T08:12:43.987 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.878994901Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-11T08:12:43.987 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.881865171Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-11T08:12:43.987 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.884755389Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-11T08:12:43.987 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.887474788Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-11T08:12:43.987 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.887901738Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-11T08:12:43.987 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.890564247Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-11T08:12:43.987 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.893147445Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-11T08:12:43.987 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.895869713Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-11T08:12:43.987 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.896461864Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-11T08:12:43.987 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.899224527Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-11T08:12:43.987 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.899552098Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-11T08:12:43.987 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.90184956Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-11T08:12:43.987 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.902462019Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-11T08:12:43.987 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.904587417Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-11T08:12:43.988 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.90493141Z level=info msg="Executing migration" id="Add uid column" 2024-09-11T08:12:43.988 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.905484899Z level=info msg="Executing migration" id="Update uid value" 2024-09-11T08:12:43.988 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.908199588Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-11T08:12:43.988 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.908658616Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-11T08:12:43.988 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.910962613Z level=info msg="Executing migration" id="create api_key table" 2024-09-11T08:12:43.988 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.91366069Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-11T08:12:43.988 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.914101971Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-11T08:12:43.988 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.914956684Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-11T08:12:43.988 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.915737897Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-11T08:12:43.988 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.916591006Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-11T08:12:43.988 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.917377884Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-11T08:12:43.988 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.918176854Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-11T08:12:43.988 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.919236715Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-11T08:12:43.989 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.919628045Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-11T08:12:43.989 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.920041617Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-11T08:12:43.989 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.920492279Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-11T08:12:43.989 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.920924902Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-11T08:12:43.989 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.921256433Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-11T08:12:43.989 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.921584486Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-11T08:12:43.989 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.92184147Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-11T08:12:43.989 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.922438485Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-11T08:12:43.989 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.92294145Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-11T08:12:43.989 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.923256641Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-11T08:12:43.989 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.923759958Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-11T08:12:43.989 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.924331289Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-11T08:12:43.989 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.924717209Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-11T08:12:43.990 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.925085563Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-11T08:12:43.990 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.925478422Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-11T08:12:43.990 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.925904689Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-11T08:12:43.990 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.926341176Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-11T08:12:43.990 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.926746347Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-11T08:12:43.990 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.927051454Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-11T08:12:43.990 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.927367823Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-11T08:12:43.990 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.927902006Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-11T08:12:43.990 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.928462682Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-11T08:12:43.990 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.928746591Z level=info msg="Executing migration" id="create quota table v1" 2024-09-11T08:12:43.990 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.929151666Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-11T08:12:43.990 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.92964132Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-11T08:12:43.990 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.929917801Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-11T08:12:43.990 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.930343278Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-11T08:12:43.990 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.930771667Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-11T08:12:43.991 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.931398063Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-11T08:12:43.991 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.931680499Z level=info msg="Executing migration" id="create session table" 2024-09-11T08:12:43.991 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.932093703Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-11T08:12:43.991 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.932384352Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-11T08:12:43.991 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.932645807Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-11T08:12:43.991 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.933023269Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-11T08:12:43.991 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.933388305Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-11T08:12:43.991 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.933637734Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-11T08:12:43.991 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.933890455Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-11T08:12:43.991 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.93418223Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-11T08:12:43.991 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.93446621Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-11T08:12:43.991 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.93483946Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-11T08:12:43.991 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.937164152Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-11T08:12:43.991 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.940020414Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-11T08:12:43.992 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.942714084Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-11T08:12:43.992 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.945463897Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-11T08:12:43.992 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.948151805Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-11T08:12:43.992 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.950964802Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-11T08:12:43.992 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.953465059Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-11T08:12:43.992 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.956201263Z level=info msg="Executing migration" id="create alert table v1" 2024-09-11T08:12:43.992 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.958952656Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-11T08:12:43.992 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.961714424Z level=info msg="Executing migration" id="add index alert state" 2024-09-11T08:12:43.992 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.964462545Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-11T08:12:43.992 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.96713895Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-11T08:12:43.992 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.969784957Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-11T08:12:43.992 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.972529895Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-11T08:12:43.993 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.97534913Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-11T08:12:43.993 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.978037321Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-11T08:12:44.168 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411369 got 77309411372 for osd.2 2024-09-11T08:12:44.168 DEBUG:teuthology.parallel:result is None 2024-09-11T08:12:44.229 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.980645106Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-11T08:12:44.229 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.983327984Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-11T08:12:44.229 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.985928046Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-11T08:12:44.229 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.988503914Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-11T08:12:44.230 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.991067902Z level=info msg="Executing migration" id="Add column is_default" 2024-09-11T08:12:44.230 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.993708915Z level=info msg="Executing migration" id="Add column frequency" 2024-09-11T08:12:44.230 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.996365722Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-11T08:12:44.230 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:43 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:43.99901279Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-11T08:12:44.230 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.001592634Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-11T08:12:44.230 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.004105944Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-11T08:12:44.230 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.006841191Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-11T08:12:44.230 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.009602048Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-11T08:12:44.230 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.012123912Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-11T08:12:44.230 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.014775976Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-11T08:12:44.230 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.017142879Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-11T08:12:44.231 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.019839632Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-11T08:12:44.231 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.022525757Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-11T08:12:44.231 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.023313122Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-11T08:12:44.231 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.024117554Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-11T08:12:44.231 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.024926593Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-11T08:12:44.231 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.025742938Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-11T08:12:44.231 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.026509423Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-11T08:12:44.231 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.027318269Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-11T08:12:44.232 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.028096052Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-11T08:12:44.232 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.028923666Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-11T08:12:44.232 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.029740048Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-11T08:12:44.232 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.030535899Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-11T08:12:44.232 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.031302632Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-11T08:12:44.232 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.032101306Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-11T08:12:44.232 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.032972142Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-11T08:12:44.232 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.033764616Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-11T08:12:44.233 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.03454109Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-11T08:12:44.233 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.035328617Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-11T08:12:44.233 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.036130275Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-11T08:12:44.233 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.036956455Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-11T08:12:44.233 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.037836209Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-11T08:12:44.233 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.038667648Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-11T08:12:44.233 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.039528513Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-11T08:12:44.233 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.04037894Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-11T08:12:44.233 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.041170733Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-11T08:12:44.233 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.042747535Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-11T08:12:44.233 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.04361656Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-11T08:12:44.233 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.044473156Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-11T08:12:44.234 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.045291766Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-11T08:12:44.234 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.046110586Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-11T08:12:44.234 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.046931944Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-11T08:12:44.234 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.047818101Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-11T08:12:44.234 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.048647108Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-11T08:12:44.234 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.049447001Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-11T08:12:44.234 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.05030045Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-11T08:12:44.234 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.051131223Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-11T08:12:44.234 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.052004584Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-11T08:12:44.234 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.054835087Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-11T08:12:44.234 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.057511572Z level=info msg="Executing migration" id="Move region to single row" 2024-09-11T08:12:44.234 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.060098884Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-11T08:12:44.234 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.062880003Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-11T08:12:44.236 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.065480462Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-11T08:12:44.236 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.068172704Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-11T08:12:44.236 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.070782659Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-11T08:12:44.236 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.073558455Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-11T08:12:44.236 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.076213602Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-11T08:12:44.236 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.07895945Z level=info msg="Executing migration" id="create test_data table" 2024-09-11T08:12:44.236 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.081639611Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-11T08:12:44.236 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.08437729Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-11T08:12:44.236 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.087205699Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-11T08:12:44.236 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.089947733Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-11T08:12:44.236 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.092617879Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-11T08:12:44.237 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.095200023Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-11T08:12:44.237 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.097936463Z level=info msg="Executing migration" id="create team table" 2024-09-11T08:12:44.237 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.100644912Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-11T08:12:44.237 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.103335551Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-11T08:12:44.237 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.105992232Z level=info msg="Executing migration" id="create team member table" 2024-09-11T08:12:44.237 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.108591882Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-11T08:12:44.237 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.111340205Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-11T08:12:44.237 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.113915611Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-11T08:12:44.237 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.116591938Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-11T08:12:44.237 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.119216632Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-11T08:12:44.237 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.121832168Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-11T08:12:44.237 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.124298396Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-11T08:12:44.237 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.12698103Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-11T08:12:44.237 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.129683412Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-11T08:12:44.238 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.132441878Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-11T08:12:44.238 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.134857079Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-11T08:12:44.238 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.137512921Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-11T08:12:44.238 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.139894068Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-11T08:12:44.238 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.14257845Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-11T08:12:44.238 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.14516534Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-11T08:12:44.238 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.145967809Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-11T08:12:44.238 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.146754342Z level=info msg="Executing migration" id="create tag table" 2024-09-11T08:12:44.238 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.147608358Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-11T08:12:44.238 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.148382699Z level=info msg="Executing migration" id="create login attempt table" 2024-09-11T08:12:44.238 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.148782823Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-11T08:12:44.238 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.149568657Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-11T08:12:44.238 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.150369816Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-11T08:12:44.238 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.15227676Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-11T08:12:44.239 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.152664125Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-11T08:12:44.239 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.153458345Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-11T08:12:44.239 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.154259361Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-11T08:12:44.239 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.15501951Z level=info msg="Executing migration" id="create user auth table" 2024-09-11T08:12:44.239 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.155413435Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-11T08:12:44.239 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.156190159Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-11T08:12:44.239 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.156966869Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-11T08:12:44.239 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.157752499Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-11T08:12:44.239 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.158545367Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-11T08:12:44.239 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.159362623Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-11T08:12:44.239 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.160219737Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-11T08:12:44.239 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.161068638Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-11T08:12:44.239 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.16180987Z level=info msg="Executing migration" id="create server_lock table" 2024-09-11T08:12:44.239 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.162596368Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-11T08:12:44.239 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.163445427Z level=info msg="Executing migration" id="create user auth token table" 2024-09-11T08:12:44.240 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.164263543Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-11T08:12:44.240 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.164697165Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-11T08:12:44.240 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.165146234Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-11T08:12:44.240 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.165574007Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-11T08:12:44.240 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.166398008Z level=info msg="Executing migration" id="create cache_data table" 2024-09-11T08:12:44.240 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.166804827Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-11T08:12:44.240 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.167253613Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-11T08:12:44.240 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.167673426Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-11T08:12:44.240 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.16811173Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-11T08:12:44.240 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.168408448Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-11T08:12:44.240 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.16869228Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-11T08:12:44.241 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.169147414Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-11T08:12:44.241 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.169878884Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-11T08:12:44.241 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.170309706Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-11T08:12:44.241 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.170583269Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-11T08:12:44.241 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.170989824Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-11T08:12:44.241 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.171428688Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-11T08:12:44.241 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.171839783Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-11T08:12:44.241 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.172294149Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-11T08:12:44.241 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.173077413Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-11T08:12:44.241 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.173474396Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-11T08:12:44.241 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.173752809Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-11T08:12:44.241 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.174178186Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-11T08:12:44.241 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.17461552Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-11T08:12:44.241 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.175054708Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-11T08:12:44.242 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.175353923Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-11T08:12:44.242 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.17572793Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-11T08:12:44.242 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.176163689Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-11T08:12:44.242 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.176621874Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-11T08:12:44.242 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.177064721Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-11T08:12:44.242 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.177841552Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-11T08:12:44.242 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.178303361Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-11T08:12:44.242 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.178700442Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-11T08:12:44.242 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.182157016Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-11T08:12:44.243 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.185493251Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-11T08:12:44.243 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.185963913Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-11T08:12:44.243 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.186404276Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-11T08:12:44.243 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.187231442Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-11T08:12:44.243 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.187657782Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-11T08:12:44.243 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.188501734Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-11T08:12:44.243 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.188950487Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-11T08:12:44.243 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.189724378Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-11T08:12:44.243 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.190594708Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-11T08:12:44.243 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.191447662Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-11T08:12:44.243 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.192265388Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-11T08:12:44.243 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.19311244Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-11T08:12:44.244 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.193965163Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-11T08:12:44.244 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.194830884Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-11T08:12:44.244 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.195645519Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-11T08:12:44.244 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.196526865Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-11T08:12:44.244 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.197411067Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-11T08:12:44.244 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.198277788Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-11T08:12:44.244 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.199104797Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-11T08:12:44.244 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.199970468Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-11T08:12:44.245 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.200836551Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-11T08:12:44.245 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.201681098Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-11T08:12:44.245 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.202124829Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-11T08:12:44.245 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.202429337Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-11T08:12:44.245 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.203336594Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-11T08:12:44.245 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.204193528Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-11T08:12:44.245 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.205038941Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-11T08:12:44.245 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.207815321Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-11T08:12:44.245 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.210659837Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-11T08:12:44.245 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.213498904Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-11T08:12:44.245 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.213951986Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-11T08:12:44.245 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.216441306Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-11T08:12:44.245 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.219183239Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-11T08:12:44.246 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.222035549Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-11T08:12:44.246 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.222495015Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-11T08:12:44.246 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.225393843Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-11T08:12:44.246 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.228201613Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-11T08:12:44.246 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.23101035Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-11T08:12:44.246 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.231901636Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-11T08:12:44.505 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-mon[30677]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:44.506 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:44.506 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:44.506 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:44.506 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:44.507 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.234828664Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-11T08:12:44.507 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.237408194Z level=info msg="Executing migration" id="create alert_image table" 2024-09-11T08:12:44.507 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.240232436Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-11T08:12:44.507 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.242886162Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-11T08:12:44.508 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.245639146Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-11T08:12:44.508 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.248474229Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-11T08:12:44.508 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.24892645Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T08:12:44.508 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.249011718Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T08:12:44.508 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.251725562Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-11T08:12:44.508 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.254530083Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-11T08:12:44.508 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.25714235Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-11T08:12:44.508 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.257310913Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-11T08:12:44.508 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.257443679Z level=info msg="alerts found to migrate" alerts=0 2024-09-11T08:12:44.508 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.259851994Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-11T08:12:44.508 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.26253824Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-11T08:12:44.508 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.265216353Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-11T08:12:44.508 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.268041876Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-11T08:12:44.509 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.270679092Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-11T08:12:44.509 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.273442015Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-11T08:12:44.509 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.276156802Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-11T08:12:44.509 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.278938872Z level=info msg="Executing migration" id="create data_keys table" 2024-09-11T08:12:44.509 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.281542251Z level=info msg="Executing migration" id="create secrets table" 2024-09-11T08:12:44.509 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.284234242Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-11T08:12:44.509 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.288510298Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-11T08:12:44.509 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.291296134Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-11T08:12:44.509 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.293950326Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-11T08:12:44.510 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.298153273Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-11T08:12:44.510 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.302340099Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-11T08:12:44.510 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.303166795Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-11T08:12:44.510 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.303975812Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-11T08:12:44.510 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.304722015Z level=info msg="Executing migration" id="create permission table" 2024-09-11T08:12:44.510 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.30512393Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-11T08:12:44.510 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.30592903Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-11T08:12:44.510 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.306766427Z level=info msg="Executing migration" id="create role table" 2024-09-11T08:12:44.510 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.307616078Z level=info msg="Executing migration" id="add column display_name" 2024-09-11T08:12:44.511 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.308564104Z level=info msg="Executing migration" id="add column group_name" 2024-09-11T08:12:44.511 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.30950687Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-11T08:12:44.511 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.310351367Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-11T08:12:44.511 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.311172971Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-11T08:12:44.511 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.312002628Z level=info msg="Executing migration" id="create team role table" 2024-09-11T08:12:44.511 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.312853192Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-11T08:12:44.511 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.31366414Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-11T08:12:44.511 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.314451473Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-11T08:12:44.511 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.315276813Z level=info msg="Executing migration" id="create user role table" 2024-09-11T08:12:44.512 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.31604176Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-11T08:12:44.512 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.316897183Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-11T08:12:44.512 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.317771015Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-11T08:12:44.512 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.318598271Z level=info msg="Executing migration" id="create builtin role table" 2024-09-11T08:12:44.512 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.319478063Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-11T08:12:44.512 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.320327167Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-11T08:12:44.512 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.321154201Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-11T08:12:44.512 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.322161738Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-11T08:12:44.512 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.32301997Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-11T08:12:44.512 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.323480451Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-11T08:12:44.512 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.323891751Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-11T08:12:44.513 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.324337474Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-11T08:12:44.513 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.324732761Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-11T08:12:44.513 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.325201703Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-11T08:12:44.513 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.326192231Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-11T08:12:44.513 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.327001517Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-11T08:12:44.513 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.327455651Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-11T08:12:44.513 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.32774045Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-11T08:12:44.513 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.328182176Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-11T08:12:44.513 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.328656525Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-11T08:12:44.513 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.329049524Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-11T08:12:44.513 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.32937801Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-11T08:12:44.513 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.329723463Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-11T08:12:44.513 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.330147834Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-11T08:12:44.513 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.330602171Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-11T08:12:44.514 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.331580792Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-11T08:12:44.514 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.332366619Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-11T08:12:44.514 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.333234126Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-11T08:12:44.514 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.334071202Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-11T08:12:44.514 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.334866443Z level=info msg="Executing migration" id="add correlation config column" 2024-09-11T08:12:44.514 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.33608869Z level=info msg="Executing migration" id="create entity_events table" 2024-09-11T08:12:44.514 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.336782057Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-11T08:12:44.514 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.337650347Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:12:44.514 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.337762713Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:12:44.514 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.33842149Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:12:44.514 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.338529107Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:12:44.514 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.339182824Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-11T08:12:44.514 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.340034466Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-11T08:12:44.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.340912141Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:12:44.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.341740474Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:12:44.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.344507408Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-11T08:12:44.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.347160139Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T08:12:44.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.349951542Z level=info msg="Executing migration" id="Drop public config table" 2024-09-11T08:12:44.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.352721209Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-11T08:12:44.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.355488242Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-11T08:12:44.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.358107359Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T08:12:44.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.36076471Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-11T08:12:44.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.363480777Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-11T08:12:44.515 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.366677068Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-11T08:12:44.516 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.369412845Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-11T08:12:44.516 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.372396713Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-11T08:12:44.516 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.374878149Z level=info msg="Executing migration" id="add share column" 2024-09-11T08:12:44.516 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.377649566Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-11T08:12:44.516 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.380405406Z level=info msg="Executing migration" id="create file table" 2024-09-11T08:12:44.516 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.38317846Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-11T08:12:44.516 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.385817694Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-11T08:12:44.516 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.388596171Z level=info msg="Executing migration" id="create file_meta table" 2024-09-11T08:12:44.516 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.391290642Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-11T08:12:44.516 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.393861273Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-11T08:12:44.517 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.396588884Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-11T08:12:44.517 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.399377803Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-11T08:12:44.517 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.402056213Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-11T08:12:44.517 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.404478178Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-11T08:12:44.517 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.407211357Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-11T08:12:44.517 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.409753189Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-11T08:12:44.517 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.412546344Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-11T08:12:44.517 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.415059305Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-11T08:12:44.517 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.417660339Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-11T08:12:44.517 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.419885192Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-11T08:12:44.517 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.422514168Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-11T08:12:44.517 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.425046333Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-11T08:12:44.517 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.427568762Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-11T08:12:44.518 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.437296595Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-11T08:12:44.518 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.43992403Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-11T08:12:44.518 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.4403699Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-11T08:12:44.518 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.443800842Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-11T08:12:44.518 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.444553671Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-11T08:12:44.518 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.445340448Z level=info msg="Executing migration" id="create folder table" 2024-09-11T08:12:44.518 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.445747571Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-11T08:12:44.518 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.446633311Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-11T08:12:44.518 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.447295604Z level=info msg="Executing migration" id="Update folder title length" 2024-09-11T08:12:44.518 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.448189142Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-11T08:12:44.518 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=migrator t=2024-09-11T08:12:44.448815724Z level=info msg="migrations completed" performed=485 skipped=0 duration=654.975532ms 2024-09-11T08:12:44.518 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=sqlstore t=2024-09-11T08:12:44.449209571Z level=info msg="Created default organization" 2024-09-11T08:12:44.518 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=plugin.loader t=2024-09-11T08:12:44.505438693Z level=info msg="Plugin registered" pluginID=input 2024-09-11T08:12:44.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:44 smithi071 ceph-mon[25834]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:44.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:44 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:44.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:44 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:44.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:44 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:44.674 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:44 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:44.675 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:44 smithi071 ceph-mon[39897]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:44.675 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:44 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:44.675 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:44 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:44.675 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:44 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:44.675 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:44 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:44.812 INFO:teuthology.orchestra.run.smithi071.stdout:34359738423 2024-09-11T08:12:44.931 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=plugin.loader t=2024-09-11T08:12:44.537513993Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-11T08:12:44.931 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=plugin.loader t=2024-09-11T08:12:44.537531999Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-11T08:12:44.931 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=secrets t=2024-09-11T08:12:44.537564731Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-11T08:12:44.931 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=query_data t=2024-09-11T08:12:44.54059095Z level=info msg="Query Service initialization" 2024-09-11T08:12:44.931 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:44 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=live.push_http t=2024-09-11T08:12:44.543234778Z level=info msg="Live Push Gateway initialization" 2024-09-11T08:12:45.116 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928233998 got 231928234004 for osd.7 2024-09-11T08:12:45.116 DEBUG:teuthology.parallel:result is None 2024-09-11T08:12:45.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-mon[30677]: Deploying daemon node-exporter.a on smithi071 2024-09-11T08:12:45.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/3013409776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:12:45.681 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=infra.usagestats.collector t=2024-09-11T08:12:45.261832478Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-11T08:12:45.681 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=provisioning.datasources t=2024-09-11T08:12:45.262297027Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-11T08:12:45.682 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=provisioning.datasources t=2024-09-11T08:12:45.295724243Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-11T08:12:45.682 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=provisioning.datasources t=2024-09-11T08:12:45.327241373Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-11T08:12:45.682 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=provisioning.alerting t=2024-09-11T08:12:45.348887086Z level=info msg="starting to provision alerting" 2024-09-11T08:12:45.682 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=provisioning.alerting t=2024-09-11T08:12:45.348911198Z level=info msg="finished to provision alerting" 2024-09-11T08:12:45.682 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=grafanaStorageLogger t=2024-09-11T08:12:45.349080952Z level=info msg="storage starting" 2024-09-11T08:12:45.682 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=ngalert.state.manager t=2024-09-11T08:12:45.349109498Z level=info msg="Warming state cache for startup" 2024-09-11T08:12:45.682 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=http.server t=2024-09-11T08:12:45.350808476Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-11T08:12:45.682 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=ngalert.state.manager t=2024-09-11T08:12:45.359293276Z level=info msg="State cache has been initialized" states=0 duration=10.182661ms 2024-09-11T08:12:45.682 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=ngalert.multiorg.alertmanager t=2024-09-11T08:12:45.359323863Z level=info msg="starting MultiOrg Alertmanager" 2024-09-11T08:12:45.682 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=ticker t=2024-09-11T08:12:45.35934066Z level=info msg=starting first_tick=2024-09-11T08:12:50Z 2024-09-11T08:12:45.682 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=sqlstore.transactions t=2024-09-11T08:12:45.359359345Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:12:45.683 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=sqlstore.transactions t=2024-09-11T08:12:45.403191681Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:12:45.683 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:12:45 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a[72385]: logger=sqlstore.transactions t=2024-09-11T08:12:45.413611298Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-11T08:12:45.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:45 smithi071 ceph-mon[39897]: Deploying daemon node-exporter.a on smithi071 2024-09-11T08:12:45.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:45 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/3013409776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:12:45.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:45 smithi071 ceph-mon[25834]: Deploying daemon node-exporter.a on smithi071 2024-09-11T08:12:45.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:45 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/3013409776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:12:45.879 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738418 got 34359738423 for osd.0 2024-09-11T08:12:45.879 DEBUG:teuthology.parallel:result is None 2024-09-11T08:12:46.431 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:46 smithi071 systemd[1]: Starting Ceph node-exporter.a for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:12:46.681 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:12:46 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: ::ffff:172.21.15.196 - - [11/Sep/2024:08:12:46] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:46.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:46 smithi196 ceph-mon[30677]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:46.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:46 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:46.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:46 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:46.694 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:46 smithi071 ceph-mon[39897]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:46.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:46 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:46.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:46 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:46.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:46 smithi071 ceph-mon[25834]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:46.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:46 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:46.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:46 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:47.386 INFO:teuthology.orchestra.run.smithi071.stdout:163208757277 2024-09-11T08:12:47.665 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:47 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/2287904522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:12:47.665 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:47 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2287904522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:12:47.666 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:47 smithi071 bash[84690]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-11T08:12:47.666 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:47 smithi071 bash[84690]: Getting image source signatures 2024-09-11T08:12:47.666 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:47 smithi071 bash[84690]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-11T08:12:47.666 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:47 smithi071 bash[84690]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-11T08:12:47.666 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:47 smithi071 bash[84690]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T08:12:47.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:47 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/2287904522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:12:48.105 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757274 got 163208757277 for osd.5 2024-09-11T08:12:48.106 DEBUG:teuthology.parallel:result is None 2024-09-11T08:12:48.106 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T08:12:48.106 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:12:48.364 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:48.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:48 smithi071 ceph-mon[25834]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:48.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:48 smithi071 ceph-mon[39897]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:48.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:48 smithi196 ceph-mon[30677]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:49.437 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:49 smithi071 bash[84690]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T08:12:49.695 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:49 smithi071 bash[84690]: Writing manifest to image destination 2024-09-11T08:12:50.154 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:49 smithi071 podman[84690]: 2024-09-11 08:12:49.939033302 +0000 UTC m=+2.706880719 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T08:12:50.440 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 podman[84690]: 2024-09-11 08:12:50.154384455 +0000 UTC m=+2.922231912 container create 4837cdf81df568b22dabb6d6344081facaa8596ff4ff61dc3751cc12516442df (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:12:50.440 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 podman[84690]: 2024-09-11 08:12:50.373075092 +0000 UTC m=+3.140922513 container init 4837cdf81df568b22dabb6d6344081facaa8596ff4ff61dc3751cc12516442df (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:12:50.440 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 podman[84690]: 2024-09-11 08:12:50.377974572 +0000 UTC m=+3.145821987 container start 4837cdf81df568b22dabb6d6344081facaa8596ff4ff61dc3751cc12516442df (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:12:50.440 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.382Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T08:12:50.440 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.382Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T08:12:50.440 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.382Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T08:12:50.441 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.383Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T08:12:50.441 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.383Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T08:12:50.441 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.383Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T08:12:50.441 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T08:12:50.441 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T08:12:50.441 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T08:12:50.441 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T08:12:50.441 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T08:12:50.441 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T08:12:50.442 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T08:12:50.442 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T08:12:50.442 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T08:12:50.442 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T08:12:50.442 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T08:12:50.443 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T08:12:50.443 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T08:12:50.443 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T08:12:50.443 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T08:12:50.443 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T08:12:50.443 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T08:12:50.443 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T08:12:50.443 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T08:12:50.443 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T08:12:50.443 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T08:12:50.443 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T08:12:50.443 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T08:12:50.443 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T08:12:50.444 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T08:12:50.444 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T08:12:50.444 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T08:12:50.444 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T08:12:50.444 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T08:12:50.444 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T08:12:50.444 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T08:12:50.444 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T08:12:50.444 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T08:12:50.444 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T08:12:50.444 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T08:12:50.445 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T08:12:50.445 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T08:12:50.445 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T08:12:50.445 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T08:12:50.445 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T08:12:50.445 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T08:12:50.445 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T08:12:50.445 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T08:12:50.445 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T08:12:50.445 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T08:12:50.445 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T08:12:50.445 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a[84895]: ts=2024-09-11T08:12:50.384Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T08:12:50.694 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-mon[25834]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:50.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:50 smithi071 ceph-mon[39897]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:50.695 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 bash[84690]: 4837cdf81df568b22dabb6d6344081facaa8596ff4ff61dc3751cc12516442df 2024-09-11T08:12:50.695 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:12:50 smithi071 systemd[1]: Started Ceph node-exporter.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:12:50.859 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:50 smithi196 ceph-mon[30677]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:51.541 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:51 smithi196 bash[72664]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-11T08:12:51.689 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:12:51.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:51 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:51.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:51 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:51.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:51 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:51.690 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:51 smithi071 ceph-mon[25834]: Deploying daemon node-exporter.b on smithi196 2024-09-11T08:12:51.690 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:51 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:51.690 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:51 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:51.690 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:51 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:51.691 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:51 smithi071 ceph-mon[39897]: Deploying daemon node-exporter.b on smithi196 2024-09-11T08:12:51.696 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T08:12:51.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:51 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:51.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:51 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:51.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:51 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:51.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:51 smithi196 ceph-mon[30677]: Deploying daemon node-exporter.b on smithi196 2024-09-11T08:12:51.931 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:51 smithi196 bash[72664]: Getting image source signatures 2024-09-11T08:12:51.932 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:51 smithi196 bash[72664]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-11T08:12:51.932 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:51 smithi196 bash[72664]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-11T08:12:51.932 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:51 smithi196 bash[72664]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T08:12:52.376 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-11T08:12:51.076175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":958,"num_read_kb":812,"num_write":609,"num_write_kb":3237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":608,"ondisk_log_size":608,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":2386576,"kb_used_data":7732,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747541872,"statfs":{"total":767926730752,"available":765482876928,"internally_reserved":0,"allocated":7917568,"data_stored":4303984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002806"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799108+0000","last_change":"2024-09-11T08:11:25.100635+0000","last_active":"2024-09-11T08:11:38.799108+0000","last_peered":"2024-09-11T08:11:38.799108+0000","last_clean":"2024-09-11T08:11:38.799108+0000","last_became_active":"2024-09-11T08:11:25.100193+0000","last_became_peered":"2024-09-11T08:11:25.100193+0000","last_unstale":"2024-09-11T08:11:38.799108+0000","last_undegraded":"2024-09-11T08:11:38.799108+0000","last_fullsized":"2024-09-11T08:11:38.799108+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:34.434988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"67'9","reported_seq":32,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349477+0000","last_change":"2024-09-11T08:11:27.100029+0000","last_active":"2024-09-11T08:11:39.349477+0000","last_peered":"2024-09-11T08:11:39.349477+0000","last_clean":"2024-09-11T08:11:39.349477+0000","last_became_active":"2024-09-11T08:11:27.099558+0000","last_became_peered":"2024-09-11T08:11:27.099558+0000","last_unstale":"2024-09-11T08:11:39.349477+0000","last_undegraded":"2024-09-11T08:11:39.349477+0000","last_fullsized":"2024-09-11T08:11:39.349477+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:11:10.230931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.847725+0000","last_change":"2024-09-11T08:11:29.095963+0000","last_active":"2024-09-11T08:11:38.847725+0000","last_peered":"2024-09-11T08:11:38.847725+0000","last_clean":"2024-09-11T08:11:38.847725+0000","last_became_active":"2024-09-11T08:11:29.095380+0000","last_became_peered":"2024-09-11T08:11:29.095380+0000","last_unstale":"2024-09-11T08:11:38.847725+0000","last_undegraded":"2024-09-11T08:11:38.847725+0000","last_fullsized":"2024-09-11T08:11:38.847725+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:08:47.386100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348502+0000","last_change":"2024-09-11T08:11:31.103488+0000","last_active":"2024-09-11T08:11:39.348502+0000","last_peered":"2024-09-11T08:11:39.348502+0000","last_clean":"2024-09-11T08:11:39.348502+0000","last_became_active":"2024-09-11T08:11:31.103199+0000","last_became_peered":"2024-09-11T08:11:31.103199+0000","last_unstale":"2024-09-11T08:11:39.348502+0000","last_undegraded":"2024-09-11T08:11:39.348502+0000","last_fullsized":"2024-09-11T08:11:39.348502+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:31.209880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799208+0000","last_change":"2024-09-11T08:11:31.418360+0000","last_active":"2024-09-11T08:11:38.799208+0000","last_peered":"2024-09-11T08:11:38.799208+0000","last_clean":"2024-09-11T08:11:38.799208+0000","last_became_active":"2024-09-11T08:11:31.418008+0000","last_became_peered":"2024-09-11T08:11:31.418008+0000","last_unstale":"2024-09-11T08:11:38.799208+0000","last_undegraded":"2024-09-11T08:11:38.799208+0000","last_fullsized":"2024-09-11T08:11:38.799208+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:27:33.474111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.800180+0000","last_change":"2024-09-11T08:11:25.094465+0000","last_active":"2024-09-11T08:11:38.800180+0000","last_peered":"2024-09-11T08:11:38.800180+0000","last_clean":"2024-09-11T08:11:38.800180+0000","last_became_active":"2024-09-11T08:11:25.093300+0000","last_became_peered":"2024-09-11T08:11:25.093300+0000","last_unstale":"2024-09-11T08:11:38.800180+0000","last_undegraded":"2024-09-11T08:11:38.800180+0000","last_fullsized":"2024-09-11T08:11:38.800180+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:30:51.694455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"67'15","reported_seq":41,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798770+0000","last_change":"2024-09-11T08:11:27.092357+0000","last_active":"2024-09-11T08:11:38.798770+0000","last_peered":"2024-09-11T08:11:38.798770+0000","last_clean":"2024-09-11T08:11:38.798770+0000","last_became_active":"2024-09-11T08:11:27.091254+0000","last_became_peered":"2024-09-11T08:11:27.091254+0000","last_unstale":"2024-09-11T08:11:38.798770+0000","last_undegraded":"2024-09-11T08:11:38.798770+0000","last_fullsized":"2024-09-11T08:11:38.798770+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:46:04.817849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349541+0000","last_change":"2024-09-11T08:11:29.097340+0000","last_active":"2024-09-11T08:11:39.349541+0000","last_peered":"2024-09-11T08:11:39.349541+0000","last_clean":"2024-09-11T08:11:39.349541+0000","last_became_active":"2024-09-11T08:11:29.097020+0000","last_became_peered":"2024-09-11T08:11:29.097020+0000","last_unstale":"2024-09-11T08:11:39.349541+0000","last_undegraded":"2024-09-11T08:11:39.349541+0000","last_fullsized":"2024-09-11T08:11:39.349541+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:07:20.456890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798555+0000","last_change":"2024-09-11T08:11:25.090587+0000","last_active":"2024-09-11T08:11:38.798555+0000","last_peered":"2024-09-11T08:11:38.798555+0000","last_clean":"2024-09-11T08:11:38.798555+0000","last_became_active":"2024-09-11T08:11:25.090318+0000","last_became_peered":"2024-09-11T08:11:25.090318+0000","last_unstale":"2024-09-11T08:11:38.798555+0000","last_undegraded":"2024-09-11T08:11:38.798555+0000","last_fullsized":"2024-09-11T08:11:38.798555+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:44:53.045065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"67'9","reported_seq":32,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348835+0000","last_change":"2024-09-11T08:11:27.093690+0000","last_active":"2024-09-11T08:11:39.348835+0000","last_peered":"2024-09-11T08:11:39.348835+0000","last_clean":"2024-09-11T08:11:39.348835+0000","last_became_active":"2024-09-11T08:11:27.093447+0000","last_became_peered":"2024-09-11T08:11:27.093447+0000","last_unstale":"2024-09-11T08:11:39.348835+0000","last_undegraded":"2024-09-11T08:11:39.348835+0000","last_fullsized":"2024-09-11T08:11:39.348835+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:42:33.447941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798106+0000","last_change":"2024-09-11T08:11:29.100712+0000","last_active":"2024-09-11T08:11:38.798106+0000","last_peered":"2024-09-11T08:11:38.798106+0000","last_clean":"2024-09-11T08:11:38.798106+0000","last_became_active":"2024-09-11T08:11:29.100434+0000","last_became_peered":"2024-09-11T08:11:29.100434+0000","last_unstale":"2024-09-11T08:11:38.798106+0000","last_undegraded":"2024-09-11T08:11:38.798106+0000","last_fullsized":"2024-09-11T08:11:38.798106+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:15:38.503280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799014+0000","last_change":"2024-09-11T08:11:31.111239+0000","last_active":"2024-09-11T08:11:38.799014+0000","last_peered":"2024-09-11T08:11:38.799014+0000","last_clean":"2024-09-11T08:11:38.799014+0000","last_became_active":"2024-09-11T08:11:31.110886+0000","last_became_peered":"2024-09-11T08:11:31.110886+0000","last_unstale":"2024-09-11T08:11:38.799014+0000","last_undegraded":"2024-09-11T08:11:38.799014+0000","last_fullsized":"2024-09-11T08:11:38.799014+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:19:12.181717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798182+0000","last_change":"2024-09-11T08:11:25.097460+0000","last_active":"2024-09-11T08:11:38.798182+0000","last_peered":"2024-09-11T08:11:38.798182+0000","last_clean":"2024-09-11T08:11:38.798182+0000","last_became_active":"2024-09-11T08:11:25.096797+0000","last_became_peered":"2024-09-11T08:11:25.096797+0000","last_unstale":"2024-09-11T08:11:38.798182+0000","last_undegraded":"2024-09-11T08:11:38.798182+0000","last_fullsized":"2024-09-11T08:11:38.798182+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:07:00.506231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"67'5","reported_seq":26,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348930+0000","last_change":"2024-09-11T08:11:27.099207+0000","last_active":"2024-09-11T08:11:39.348930+0000","last_peered":"2024-09-11T08:11:39.348930+0000","last_clean":"2024-09-11T08:11:39.348930+0000","last_became_active":"2024-09-11T08:11:27.098976+0000","last_became_peered":"2024-09-11T08:11:27.098976+0000","last_unstale":"2024-09-11T08:11:39.348930+0000","last_undegraded":"2024-09-11T08:11:39.348930+0000","last_fullsized":"2024-09-11T08:11:39.348930+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:52.194448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348696+0000","last_change":"2024-09-11T08:11:29.095767+0000","last_active":"2024-09-11T08:11:39.348696+0000","last_peered":"2024-09-11T08:11:39.348696+0000","last_clean":"2024-09-11T08:11:39.348696+0000","last_became_active":"2024-09-11T08:11:29.095486+0000","last_became_peered":"2024-09-11T08:11:29.095486+0000","last_unstale":"2024-09-11T08:11:39.348696+0000","last_undegraded":"2024-09-11T08:11:39.348696+0000","last_fullsized":"2024-09-11T08:11:39.348696+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:19:18.172557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.797987+0000","last_change":"2024-09-11T08:11:31.106927+0000","last_active":"2024-09-11T08:11:38.797987+0000","last_peered":"2024-09-11T08:11:38.797987+0000","last_clean":"2024-09-11T08:11:38.797987+0000","last_became_active":"2024-09-11T08:11:31.106638+0000","last_became_peered":"2024-09-11T08:11:31.106638+0000","last_unstale":"2024-09-11T08:11:38.797987+0000","last_undegraded":"2024-09-11T08:11:38.797987+0000","last_fullsized":"2024-09-11T08:11:38.797987+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:05:19.586061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349464+0000","last_change":"2024-09-11T08:11:31.416861+0000","last_active":"2024-09-11T08:11:39.349464+0000","last_peered":"2024-09-11T08:11:39.349464+0000","last_clean":"2024-09-11T08:11:39.349464+0000","last_became_active":"2024-09-11T08:11:31.416214+0000","last_became_peered":"2024-09-11T08:11:31.416214+0000","last_unstale":"2024-09-11T08:11:39.349464+0000","last_undegraded":"2024-09-11T08:11:39.349464+0000","last_fullsized":"2024-09-11T08:11:39.349464+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:31:05.089040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799845+0000","last_change":"2024-09-11T08:11:25.100616+0000","last_active":"2024-09-11T08:11:38.799845+0000","last_peered":"2024-09-11T08:11:38.799845+0000","last_clean":"2024-09-11T08:11:38.799845+0000","last_became_active":"2024-09-11T08:11:25.099718+0000","last_became_peered":"2024-09-11T08:11:25.099718+0000","last_unstale":"2024-09-11T08:11:38.799845+0000","last_undegraded":"2024-09-11T08:11:38.799845+0000","last_fullsized":"2024-09-11T08:11:38.799845+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:01:55.547509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":41,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349331+0000","last_change":"2024-09-11T08:11:27.085676+0000","last_active":"2024-09-11T08:11:39.349331+0000","last_peered":"2024-09-11T08:11:39.349331+0000","last_clean":"2024-09-11T08:11:39.349331+0000","last_became_active":"2024-09-11T08:11:27.085398+0000","last_became_peered":"2024-09-11T08:11:27.085398+0000","last_unstale":"2024-09-11T08:11:39.349331+0000","last_undegraded":"2024-09-11T08:11:39.349331+0000","last_fullsized":"2024-09-11T08:11:39.349331+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:21.142564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.848283+0000","last_change":"2024-09-11T08:11:29.100692+0000","last_active":"2024-09-11T08:11:38.848283+0000","last_peered":"2024-09-11T08:11:38.848283+0000","last_clean":"2024-09-11T08:11:38.848283+0000","last_became_active":"2024-09-11T08:11:29.100287+0000","last_became_peered":"2024-09-11T08:11:29.100287+0000","last_unstale":"2024-09-11T08:11:38.848283+0000","last_undegraded":"2024-09-11T08:11:38.848283+0000","last_fullsized":"2024-09-11T08:11:38.848283+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:08:00.996144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.800362+0000","last_change":"2024-09-11T08:11:31.418085+0000","last_active":"2024-09-11T08:11:38.800362+0000","last_peered":"2024-09-11T08:11:38.800362+0000","last_clean":"2024-09-11T08:11:38.800362+0000","last_became_active":"2024-09-11T08:11:31.416348+0000","last_became_peered":"2024-09-11T08:11:31.416348+0000","last_unstale":"2024-09-11T08:11:38.800362+0000","last_undegraded":"2024-09-11T08:11:38.800362+0000","last_fullsized":"2024-09-11T08:11:38.800362+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:27:22.014521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349254+0000","last_change":"2024-09-11T08:11:25.091739+0000","last_active":"2024-09-11T08:11:39.349254+0000","last_peered":"2024-09-11T08:11:39.349254+0000","last_clean":"2024-09-11T08:11:39.349254+0000","last_became_active":"2024-09-11T08:11:25.091475+0000","last_became_peered":"2024-09-11T08:11:25.091475+0000","last_unstale":"2024-09-11T08:11:39.349254+0000","last_undegraded":"2024-09-11T08:11:39.349254+0000","last_fullsized":"2024-09-11T08:11:39.349254+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:05:39.749321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"67'12","reported_seq":39,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799797+0000","last_change":"2024-09-11T08:11:27.094286+0000","last_active":"2024-09-11T08:11:38.799797+0000","last_peered":"2024-09-11T08:11:38.799797+0000","last_clean":"2024-09-11T08:11:38.799797+0000","last_became_active":"2024-09-11T08:11:27.093647+0000","last_became_peered":"2024-09-11T08:11:27.093647+0000","last_unstale":"2024-09-11T08:11:38.799797+0000","last_undegraded":"2024-09-11T08:11:38.799797+0000","last_fullsized":"2024-09-11T08:11:38.799797+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:24:11.420124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348923+0000","last_change":"2024-09-11T08:11:29.092335+0000","last_active":"2024-09-11T08:11:39.348923+0000","last_peered":"2024-09-11T08:11:39.348923+0000","last_clean":"2024-09-11T08:11:39.348923+0000","last_became_active":"2024-09-11T08:11:29.092041+0000","last_became_peered":"2024-09-11T08:11:29.092041+0000","last_unstale":"2024-09-11T08:11:39.348923+0000","last_undegraded":"2024-09-11T08:11:39.348923+0000","last_fullsized":"2024-09-11T08:11:39.348923+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:44:47.268537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"67'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348418+0000","last_change":"2024-09-11T08:11:31.109287+0000","last_active":"2024-09-11T08:11:39.348418+0000","last_peered":"2024-09-11T08:11:39.348418+0000","last_clean":"2024-09-11T08:11:39.348418+0000","last_became_active":"2024-09-11T08:11:31.108998+0000","last_became_peered":"2024-09-11T08:11:31.108998+0000","last_unstale":"2024-09-11T08:11:39.348418+0000","last_undegraded":"2024-09-11T08:11:39.348418+0000","last_fullsized":"2024-09-11T08:11:39.348418+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:15.748936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.847982+0000","last_change":"2024-09-11T08:11:25.085876+0000","last_active":"2024-09-11T08:11:38.847982+0000","last_peered":"2024-09-11T08:11:38.847982+0000","last_clean":"2024-09-11T08:11:38.847982+0000","last_became_active":"2024-09-11T08:11:25.085605+0000","last_became_peered":"2024-09-11T08:11:25.085605+0000","last_unstale":"2024-09-11T08:11:38.847982+0000","last_undegraded":"2024-09-11T08:11:38.847982+0000","last_fullsized":"2024-09-11T08:11:38.847982+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:58:58.546112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"67'10","reported_seq":31,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799589+0000","last_change":"2024-09-11T08:11:27.094925+0000","last_active":"2024-09-11T08:11:38.799589+0000","last_peered":"2024-09-11T08:11:38.799589+0000","last_clean":"2024-09-11T08:11:38.799589+0000","last_became_active":"2024-09-11T08:11:27.094637+0000","last_became_peered":"2024-09-11T08:11:27.094637+0000","last_unstale":"2024-09-11T08:11:38.799589+0000","last_undegraded":"2024-09-11T08:11:38.799589+0000","last_fullsized":"2024-09-11T08:11:38.799589+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:47.132046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"67'8","reported_seq":41,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:38.407435+0000","last_change":"2024-09-11T08:11:29.100326+0000","last_active":"2024-09-11T08:12:38.407435+0000","last_peered":"2024-09-11T08:12:38.407435+0000","last_clean":"2024-09-11T08:12:38.407435+0000","last_became_active":"2024-09-11T08:11:29.100000+0000","last_became_peered":"2024-09-11T08:11:29.100000+0000","last_unstale":"2024-09-11T08:12:38.407435+0000","last_undegraded":"2024-09-11T08:12:38.407435+0000","last_fullsized":"2024-09-11T08:12:38.407435+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:46.095412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":41,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.848552+0000","last_change":"2024-09-11T08:11:27.099619+0000","last_active":"2024-09-11T08:11:38.848552+0000","last_peered":"2024-09-11T08:11:38.848552+0000","last_clean":"2024-09-11T08:11:38.848552+0000","last_became_active":"2024-09-11T08:11:27.099102+0000","last_became_peered":"2024-09-11T08:11:27.099102+0000","last_unstale":"2024-09-11T08:11:38.848552+0000","last_undegraded":"2024-09-11T08:11:38.848552+0000","last_fullsized":"2024-09-11T08:11:38.848552+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:48.511606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799396+0000","last_change":"2024-09-11T08:11:25.097018+0000","last_active":"2024-09-11T08:11:38.799396+0000","last_peered":"2024-09-11T08:11:38.799396+0000","last_clean":"2024-09-11T08:11:38.799396+0000","last_became_active":"2024-09-11T08:11:25.096771+0000","last_became_peered":"2024-09-11T08:11:25.096771+0000","last_unstale":"2024-09-11T08:11:38.799396+0000","last_undegraded":"2024-09-11T08:11:38.799396+0000","last_fullsized":"2024-09-11T08:11:38.799396+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:17:49.864799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":38,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:38.407436+0000","last_change":"2024-09-11T08:11:29.102472+0000","last_active":"2024-09-11T08:12:38.407436+0000","last_peered":"2024-09-11T08:12:38.407436+0000","last_clean":"2024-09-11T08:12:38.407436+0000","last_became_active":"2024-09-11T08:11:29.101756+0000","last_became_peered":"2024-09-11T08:11:29.101756+0000","last_unstale":"2024-09-11T08:12:38.407436+0000","last_undegraded":"2024-09-11T08:12:38.407436+0000","last_fullsized":"2024-09-11T08:12:38.407436+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:58:44.331961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798190+0000","last_change":"2024-09-11T08:11:31.108596+0000","last_active":"2024-09-11T08:11:38.798190+0000","last_peered":"2024-09-11T08:11:38.798190+0000","last_clean":"2024-09-11T08:11:38.798190+0000","last_became_active":"2024-09-11T08:11:31.108272+0000","last_became_peered":"2024-09-11T08:11:31.108272+0000","last_unstale":"2024-09-11T08:11:38.798190+0000","last_undegraded":"2024-09-11T08:11:38.798190+0000","last_fullsized":"2024-09-11T08:11:38.798190+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:01:04.360030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"67'18","reported_seq":48,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799182+0000","last_change":"2024-09-11T08:11:27.093207+0000","last_active":"2024-09-11T08:11:38.799182+0000","last_peered":"2024-09-11T08:11:38.799182+0000","last_clean":"2024-09-11T08:11:38.799182+0000","last_became_active":"2024-09-11T08:11:27.091391+0000","last_became_peered":"2024-09-11T08:11:27.091391+0000","last_unstale":"2024-09-11T08:11:38.799182+0000","last_undegraded":"2024-09-11T08:11:38.799182+0000","last_fullsized":"2024-09-11T08:11:38.799182+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:31:52.841914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.800478+0000","last_change":"2024-09-11T08:11:25.095035+0000","last_active":"2024-09-11T08:11:38.800478+0000","last_peered":"2024-09-11T08:11:38.800478+0000","last_clean":"2024-09-11T08:11:38.800478+0000","last_became_active":"2024-09-11T08:11:25.094075+0000","last_became_peered":"2024-09-11T08:11:25.094075+0000","last_unstale":"2024-09-11T08:11:38.800478+0000","last_undegraded":"2024-09-11T08:11:38.800478+0000","last_fullsized":"2024-09-11T08:11:38.800478+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:23.648793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349800+0000","last_change":"2024-09-11T08:11:29.091792+0000","last_active":"2024-09-11T08:11:39.349800+0000","last_peered":"2024-09-11T08:11:39.349800+0000","last_clean":"2024-09-11T08:11:39.349800+0000","last_became_active":"2024-09-11T08:11:29.091483+0000","last_became_peered":"2024-09-11T08:11:29.091483+0000","last_unstale":"2024-09-11T08:11:39.349800+0000","last_undegraded":"2024-09-11T08:11:39.349800+0000","last_fullsized":"2024-09-11T08:11:39.349800+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:17.519130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348399+0000","last_change":"2024-09-11T08:11:31.106932+0000","last_active":"2024-09-11T08:11:39.348399+0000","last_peered":"2024-09-11T08:11:39.348399+0000","last_clean":"2024-09-11T08:11:39.348399+0000","last_became_active":"2024-09-11T08:11:31.105616+0000","last_became_peered":"2024-09-11T08:11:31.105616+0000","last_unstale":"2024-09-11T08:11:39.348399+0000","last_undegraded":"2024-09-11T08:11:39.348399+0000","last_fullsized":"2024-09-11T08:11:39.348399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:44:44.660887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"67'14","reported_seq":37,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349887+0000","last_change":"2024-09-11T08:11:27.092287+0000","last_active":"2024-09-11T08:11:39.349887+0000","last_peered":"2024-09-11T08:11:39.349887+0000","last_clean":"2024-09-11T08:11:39.349887+0000","last_became_active":"2024-09-11T08:11:27.091956+0000","last_became_peered":"2024-09-11T08:11:27.091956+0000","last_unstale":"2024-09-11T08:11:39.349887+0000","last_undegraded":"2024-09-11T08:11:39.349887+0000","last_fullsized":"2024-09-11T08:11:39.349887+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:11:37.593994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"60'1","reported_seq":27,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799845+0000","last_change":"2024-09-11T08:11:25.097889+0000","last_active":"2024-09-11T08:11:38.799845+0000","last_peered":"2024-09-11T08:11:38.799845+0000","last_clean":"2024-09-11T08:11:38.799845+0000","last_became_active":"2024-09-11T08:11:25.096358+0000","last_became_peered":"2024-09-11T08:11:25.096358+0000","last_unstale":"2024-09-11T08:11:38.799845+0000","last_undegraded":"2024-09-11T08:11:38.799845+0000","last_fullsized":"2024-09-11T08:11:38.799845+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:21.009658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":38,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:38.407369+0000","last_change":"2024-09-11T08:11:29.096174+0000","last_active":"2024-09-11T08:12:38.407369+0000","last_peered":"2024-09-11T08:12:38.407369+0000","last_clean":"2024-09-11T08:12:38.407369+0000","last_became_active":"2024-09-11T08:11:29.095893+0000","last_became_peered":"2024-09-11T08:11:29.095893+0000","last_unstale":"2024-09-11T08:12:38.407369+0000","last_undegraded":"2024-09-11T08:12:38.407369+0000","last_fullsized":"2024-09-11T08:12:38.407369+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:36:14.701080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348974+0000","last_change":"2024-09-11T08:11:31.418326+0000","last_active":"2024-09-11T08:11:39.348974+0000","last_peered":"2024-09-11T08:11:39.348974+0000","last_clean":"2024-09-11T08:11:39.348974+0000","last_became_active":"2024-09-11T08:11:31.417976+0000","last_became_peered":"2024-09-11T08:11:31.417976+0000","last_unstale":"2024-09-11T08:11:39.348974+0000","last_undegraded":"2024-09-11T08:11:39.348974+0000","last_fullsized":"2024-09-11T08:11:39.348974+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:34:48.447713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"67'10","reported_seq":31,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.848106+0000","last_change":"2024-09-11T08:11:27.094719+0000","last_active":"2024-09-11T08:11:38.848106+0000","last_peered":"2024-09-11T08:11:38.848106+0000","last_clean":"2024-09-11T08:11:38.848106+0000","last_became_active":"2024-09-11T08:11:27.093081+0000","last_became_peered":"2024-09-11T08:11:27.093081+0000","last_unstale":"2024-09-11T08:11:38.848106+0000","last_undegraded":"2024-09-11T08:11:38.848106+0000","last_fullsized":"2024-09-11T08:11:38.848106+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:28:27.045201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798617+0000","last_change":"2024-09-11T08:11:25.089227+0000","last_active":"2024-09-11T08:11:38.798617+0000","last_peered":"2024-09-11T08:11:38.798617+0000","last_clean":"2024-09-11T08:11:38.798617+0000","last_became_active":"2024-09-11T08:11:25.088974+0000","last_became_peered":"2024-09-11T08:11:25.088974+0000","last_unstale":"2024-09-11T08:11:38.798617+0000","last_undegraded":"2024-09-11T08:11:38.798617+0000","last_fullsized":"2024-09-11T08:11:38.798617+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:12:49.783167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"67'8","reported_seq":38,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:38.407347+0000","last_change":"2024-09-11T08:11:29.099049+0000","last_active":"2024-09-11T08:12:38.407347+0000","last_peered":"2024-09-11T08:12:38.407347+0000","last_clean":"2024-09-11T08:12:38.407347+0000","last_became_active":"2024-09-11T08:11:29.098760+0000","last_became_peered":"2024-09-11T08:11:29.098760+0000","last_unstale":"2024-09-11T08:12:38.407347+0000","last_undegraded":"2024-09-11T08:12:38.407347+0000","last_fullsized":"2024-09-11T08:12:38.407347+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:11:46.784852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798598+0000","last_change":"2024-09-11T08:11:31.114406+0000","last_active":"2024-09-11T08:11:38.798598+0000","last_peered":"2024-09-11T08:11:38.798598+0000","last_clean":"2024-09-11T08:11:38.798598+0000","last_became_active":"2024-09-11T08:11:31.114102+0000","last_became_peered":"2024-09-11T08:11:31.114102+0000","last_unstale":"2024-09-11T08:11:38.798598+0000","last_undegraded":"2024-09-11T08:11:38.798598+0000","last_fullsized":"2024-09-11T08:11:38.798598+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:46:04.586606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"67'19","reported_seq":52,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799762+0000","last_change":"2024-09-11T08:11:27.091494+0000","last_active":"2024-09-11T08:11:38.799762+0000","last_peered":"2024-09-11T08:11:38.799762+0000","last_clean":"2024-09-11T08:11:38.799762+0000","last_became_active":"2024-09-11T08:11:27.091145+0000","last_became_peered":"2024-09-11T08:11:27.091145+0000","last_unstale":"2024-09-11T08:11:38.799762+0000","last_undegraded":"2024-09-11T08:11:38.799762+0000","last_fullsized":"2024-09-11T08:11:38.799762+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:29:37.648590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.848451+0000","last_change":"2024-09-11T08:11:25.096402+0000","last_active":"2024-09-11T08:11:38.848451+0000","last_peered":"2024-09-11T08:11:38.848451+0000","last_clean":"2024-09-11T08:11:38.848451+0000","last_became_active":"2024-09-11T08:11:25.096135+0000","last_became_peered":"2024-09-11T08:11:25.096135+0000","last_unstale":"2024-09-11T08:11:38.848451+0000","last_undegraded":"2024-09-11T08:11:38.848451+0000","last_fullsized":"2024-09-11T08:11:38.848451+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:18.478984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.808199+0000","last_change":"2024-09-11T08:11:29.099261+0000","last_active":"2024-09-11T08:11:38.808199+0000","last_peered":"2024-09-11T08:11:38.808199+0000","last_clean":"2024-09-11T08:11:38.808199+0000","last_became_active":"2024-09-11T08:11:29.098974+0000","last_became_peered":"2024-09-11T08:11:29.098974+0000","last_unstale":"2024-09-11T08:11:38.808199+0000","last_undegraded":"2024-09-11T08:11:38.808199+0000","last_fullsized":"2024-09-11T08:11:38.808199+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:47:51.029612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.847668+0000","last_change":"2024-09-11T08:11:31.416917+0000","last_active":"2024-09-11T08:11:38.847668+0000","last_peered":"2024-09-11T08:11:38.847668+0000","last_clean":"2024-09-11T08:11:38.847668+0000","last_became_active":"2024-09-11T08:11:31.416538+0000","last_became_peered":"2024-09-11T08:11:31.416538+0000","last_unstale":"2024-09-11T08:11:38.847668+0000","last_undegraded":"2024-09-11T08:11:38.847668+0000","last_fullsized":"2024-09-11T08:11:38.847668+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:38:14.247963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"67'28","reported_seq":80,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:38.407344+0000","last_change":"2024-09-11T08:11:27.097248+0000","last_active":"2024-09-11T08:12:38.407344+0000","last_peered":"2024-09-11T08:12:38.407344+0000","last_clean":"2024-09-11T08:12:38.407344+0000","last_became_active":"2024-09-11T08:11:27.096947+0000","last_became_peered":"2024-09-11T08:11:27.096947+0000","last_unstale":"2024-09-11T08:12:38.407344+0000","last_undegraded":"2024-09-11T08:12:38.407344+0000","last_fullsized":"2024-09-11T08:12:38.407344+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:51.723234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349228+0000","last_change":"2024-09-11T08:11:25.093960+0000","last_active":"2024-09-11T08:11:39.349228+0000","last_peered":"2024-09-11T08:11:39.349228+0000","last_clean":"2024-09-11T08:11:39.349228+0000","last_became_active":"2024-09-11T08:11:25.093461+0000","last_became_peered":"2024-09-11T08:11:25.093461+0000","last_unstale":"2024-09-11T08:11:39.349228+0000","last_undegraded":"2024-09-11T08:11:39.349228+0000","last_fullsized":"2024-09-11T08:11:39.349228+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:36.271963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"64'2","reported_seq":25,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798430+0000","last_change":"2024-09-11T08:11:29.091587+0000","last_active":"2024-09-11T08:11:38.798430+0000","last_peered":"2024-09-11T08:11:38.798430+0000","last_clean":"2024-09-11T08:11:38.798430+0000","last_became_active":"2024-09-11T08:11:25.096910+0000","last_became_peered":"2024-09-11T08:11:25.096910+0000","last_unstale":"2024-09-11T08:11:38.798430+0000","last_undegraded":"2024-09-11T08:11:38.798430+0000","last_fullsized":"2024-09-11T08:11:38.798430+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:03.774605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001668763,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798973+0000","last_change":"2024-09-11T08:11:29.099919+0000","last_active":"2024-09-11T08:11:38.798973+0000","last_peered":"2024-09-11T08:11:38.798973+0000","last_clean":"2024-09-11T08:11:38.798973+0000","last_became_active":"2024-09-11T08:11:29.099262+0000","last_became_peered":"2024-09-11T08:11:29.099262+0000","last_unstale":"2024-09-11T08:11:38.798973+0000","last_undegraded":"2024-09-11T08:11:38.798973+0000","last_fullsized":"2024-09-11T08:11:38.798973+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:21:38.486568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.800074+0000","last_change":"2024-09-11T08:11:31.105837+0000","last_active":"2024-09-11T08:11:38.800074+0000","last_peered":"2024-09-11T08:11:38.800074+0000","last_clean":"2024-09-11T08:11:38.800074+0000","last_became_active":"2024-09-11T08:11:31.105565+0000","last_became_peered":"2024-09-11T08:11:31.105565+0000","last_unstale":"2024-09-11T08:11:38.800074+0000","last_undegraded":"2024-09-11T08:11:38.800074+0000","last_fullsized":"2024-09-11T08:11:38.800074+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:09:17.153498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"67'13","reported_seq":43,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.848540+0000","last_change":"2024-09-11T08:11:27.095117+0000","last_active":"2024-09-11T08:11:38.848540+0000","last_peered":"2024-09-11T08:11:38.848540+0000","last_clean":"2024-09-11T08:11:38.848540+0000","last_became_active":"2024-09-11T08:11:27.094824+0000","last_became_peered":"2024-09-11T08:11:27.094824+0000","last_unstale":"2024-09-11T08:11:38.848540+0000","last_undegraded":"2024-09-11T08:11:38.848540+0000","last_fullsized":"2024-09-11T08:11:38.848540+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:15.592489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.848040+0000","last_change":"2024-09-11T08:11:25.092824+0000","last_active":"2024-09-11T08:11:38.848040+0000","last_peered":"2024-09-11T08:11:38.848040+0000","last_clean":"2024-09-11T08:11:38.848040+0000","last_became_active":"2024-09-11T08:11:25.092401+0000","last_became_peered":"2024-09-11T08:11:25.092401+0000","last_unstale":"2024-09-11T08:11:38.848040+0000","last_undegraded":"2024-09-11T08:11:38.848040+0000","last_fullsized":"2024-09-11T08:11:38.848040+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:54.500772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"62'1","reported_seq":24,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348726+0000","last_change":"2024-09-11T08:11:29.082405+0000","last_active":"2024-09-11T08:11:39.348726+0000","last_peered":"2024-09-11T08:11:39.348726+0000","last_clean":"2024-09-11T08:11:39.348726+0000","last_became_active":"2024-09-11T08:11:25.094366+0000","last_became_peered":"2024-09-11T08:11:25.094366+0000","last_unstale":"2024-09-11T08:11:39.348726+0000","last_undegraded":"2024-09-11T08:11:39.348726+0000","last_fullsized":"2024-09-11T08:11:39.348726+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:41:06.586281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000367742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349220+0000","last_change":"2024-09-11T08:11:29.096678+0000","last_active":"2024-09-11T08:11:39.349220+0000","last_peered":"2024-09-11T08:11:39.349220+0000","last_clean":"2024-09-11T08:11:39.349220+0000","last_became_active":"2024-09-11T08:11:29.096397+0000","last_became_peered":"2024-09-11T08:11:29.096397+0000","last_unstale":"2024-09-11T08:11:39.349220+0000","last_undegraded":"2024-09-11T08:11:39.349220+0000","last_fullsized":"2024-09-11T08:11:39.349220+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:26.149855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349068+0000","last_change":"2024-09-11T08:11:31.416782+0000","last_active":"2024-09-11T08:11:39.349068+0000","last_peered":"2024-09-11T08:11:39.349068+0000","last_clean":"2024-09-11T08:11:39.349068+0000","last_became_active":"2024-09-11T08:11:31.416481+0000","last_became_peered":"2024-09-11T08:11:31.416481+0000","last_unstale":"2024-09-11T08:11:39.349068+0000","last_undegraded":"2024-09-11T08:11:39.349068+0000","last_fullsized":"2024-09-11T08:11:39.349068+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:39:09.994375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"67'12","reported_seq":34,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799672+0000","last_change":"2024-09-11T08:11:27.096351+0000","last_active":"2024-09-11T08:11:38.799672+0000","last_peered":"2024-09-11T08:11:38.799672+0000","last_clean":"2024-09-11T08:11:38.799672+0000","last_became_active":"2024-09-11T08:11:27.096081+0000","last_became_peered":"2024-09-11T08:11:27.096081+0000","last_unstale":"2024-09-11T08:11:38.799672+0000","last_undegraded":"2024-09-11T08:11:38.799672+0000","last_fullsized":"2024-09-11T08:11:38.799672+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:55:58.950174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798710+0000","last_change":"2024-09-11T08:11:25.098694+0000","last_active":"2024-09-11T08:11:38.798710+0000","last_peered":"2024-09-11T08:11:38.798710+0000","last_clean":"2024-09-11T08:11:38.798710+0000","last_became_active":"2024-09-11T08:11:25.098070+0000","last_became_peered":"2024-09-11T08:11:25.098070+0000","last_unstale":"2024-09-11T08:11:38.798710+0000","last_undegraded":"2024-09-11T08:11:38.798710+0000","last_fullsized":"2024-09-11T08:11:38.798710+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:00:49.357459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":101,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:45.772882+0000","last_change":"2024-09-11T08:11:29.089234+0000","last_active":"2024-09-11T08:12:45.772882+0000","last_peered":"2024-09-11T08:12:45.772882+0000","last_clean":"2024-09-11T08:12:45.772882+0000","last_became_active":"2024-09-11T08:11:25.094995+0000","last_became_peered":"2024-09-11T08:11:25.094995+0000","last_unstale":"2024-09-11T08:12:45.772882+0000","last_undegraded":"2024-09-11T08:12:45.772882+0000","last_fullsized":"2024-09-11T08:12:45.772882+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:43:19.613394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000316562,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798610+0000","last_change":"2024-09-11T08:11:29.099690+0000","last_active":"2024-09-11T08:11:38.798610+0000","last_peered":"2024-09-11T08:11:38.798610+0000","last_clean":"2024-09-11T08:11:38.798610+0000","last_became_active":"2024-09-11T08:11:29.099315+0000","last_became_peered":"2024-09-11T08:11:29.099315+0000","last_unstale":"2024-09-11T08:11:38.798610+0000","last_undegraded":"2024-09-11T08:11:38.798610+0000","last_fullsized":"2024-09-11T08:11:38.798610+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:52.867239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.847853+0000","last_change":"2024-09-11T08:11:31.106743+0000","last_active":"2024-09-11T08:11:38.847853+0000","last_peered":"2024-09-11T08:11:38.847853+0000","last_clean":"2024-09-11T08:11:38.847853+0000","last_became_active":"2024-09-11T08:11:31.106471+0000","last_became_peered":"2024-09-11T08:11:31.106471+0000","last_unstale":"2024-09-11T08:11:38.847853+0000","last_undegraded":"2024-09-11T08:11:38.847853+0000","last_fullsized":"2024-09-11T08:11:38.847853+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:52:16.190282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":38,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.808621+0000","last_change":"2024-09-11T08:11:27.089055+0000","last_active":"2024-09-11T08:11:38.808621+0000","last_peered":"2024-09-11T08:11:38.808621+0000","last_clean":"2024-09-11T08:11:38.808621+0000","last_became_active":"2024-09-11T08:11:27.088754+0000","last_became_peered":"2024-09-11T08:11:27.088754+0000","last_unstale":"2024-09-11T08:11:38.808621+0000","last_undegraded":"2024-09-11T08:11:38.808621+0000","last_fullsized":"2024-09-11T08:11:38.808621+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:49:34.528176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799094+0000","last_change":"2024-09-11T08:11:25.091612+0000","last_active":"2024-09-11T08:11:38.799094+0000","last_peered":"2024-09-11T08:11:38.799094+0000","last_clean":"2024-09-11T08:11:38.799094+0000","last_became_active":"2024-09-11T08:11:25.091216+0000","last_became_peered":"2024-09-11T08:11:25.091216+0000","last_unstale":"2024-09-11T08:11:38.799094+0000","last_undegraded":"2024-09-11T08:11:38.799094+0000","last_fullsized":"2024-09-11T08:11:38.799094+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:43:14.681940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"69'139","reported_seq":271,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:03.115868+0000","last_change":"2024-09-11T08:11:16.050351+0000","last_active":"2024-09-11T08:12:03.115868+0000","last_peered":"2024-09-11T08:12:03.115868+0000","last_clean":"2024-09-11T08:12:03.115868+0000","last_became_active":"2024-09-11T08:11:16.049986+0000","last_became_peered":"2024-09-11T08:11:16.049986+0000","last_unstale":"2024-09-11T08:12:03.115868+0000","last_undegraded":"2024-09-11T08:12:03.115868+0000","last_fullsized":"2024-09-11T08:12:03.115868+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:09:13.980611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:09:13.980611+0000","last_clean_scrub_stamp":"2024-09-11T08:09:13.980611+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:36:46.350373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":258,"num_write":179,"num_write_kb":3194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349570+0000","last_change":"2024-09-11T08:11:29.095149+0000","last_active":"2024-09-11T08:11:39.349570+0000","last_peered":"2024-09-11T08:11:39.349570+0000","last_clean":"2024-09-11T08:11:39.349570+0000","last_became_active":"2024-09-11T08:11:29.094835+0000","last_became_peered":"2024-09-11T08:11:29.094835+0000","last_unstale":"2024-09-11T08:11:39.349570+0000","last_undegraded":"2024-09-11T08:11:39.349570+0000","last_fullsized":"2024-09-11T08:11:39.349570+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:03.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798647+0000","last_change":"2024-09-11T08:11:31.105546+0000","last_active":"2024-09-11T08:11:38.798647+0000","last_peered":"2024-09-11T08:11:38.798647+0000","last_clean":"2024-09-11T08:11:38.798647+0000","last_became_active":"2024-09-11T08:11:31.105285+0000","last_became_peered":"2024-09-11T08:11:31.105285+0000","last_unstale":"2024-09-11T08:11:38.798647+0000","last_undegraded":"2024-09-11T08:11:38.798647+0000","last_fullsized":"2024-09-11T08:11:38.798647+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:03:01.327591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"67'11","reported_seq":35,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349620+0000","last_change":"2024-09-11T08:11:27.099851+0000","last_active":"2024-09-11T08:11:39.349620+0000","last_peered":"2024-09-11T08:11:39.349620+0000","last_clean":"2024-09-11T08:11:39.349620+0000","last_became_active":"2024-09-11T08:11:27.098448+0000","last_became_peered":"2024-09-11T08:11:27.098448+0000","last_unstale":"2024-09-11T08:11:39.349620+0000","last_undegraded":"2024-09-11T08:11:39.349620+0000","last_fullsized":"2024-09-11T08:11:39.349620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:40.555678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.350018+0000","last_change":"2024-09-11T08:11:25.090100+0000","last_active":"2024-09-11T08:11:39.350018+0000","last_peered":"2024-09-11T08:11:39.350018+0000","last_clean":"2024-09-11T08:11:39.350018+0000","last_became_active":"2024-09-11T08:11:25.089810+0000","last_became_peered":"2024-09-11T08:11:25.089810+0000","last_unstale":"2024-09-11T08:11:39.350018+0000","last_undegraded":"2024-09-11T08:11:39.350018+0000","last_fullsized":"2024-09-11T08:11:39.350018+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:00:15.181451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798476+0000","last_change":"2024-09-11T08:11:29.099677+0000","last_active":"2024-09-11T08:11:38.798476+0000","last_peered":"2024-09-11T08:11:38.798476+0000","last_clean":"2024-09-11T08:11:38.798476+0000","last_became_active":"2024-09-11T08:11:29.099339+0000","last_became_peered":"2024-09-11T08:11:29.099339+0000","last_unstale":"2024-09-11T08:11:38.798476+0000","last_undegraded":"2024-09-11T08:11:38.798476+0000","last_fullsized":"2024-09-11T08:11:38.798476+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:38.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799625+0000","last_change":"2024-09-11T08:11:31.416874+0000","last_active":"2024-09-11T08:11:38.799625+0000","last_peered":"2024-09-11T08:11:38.799625+0000","last_clean":"2024-09-11T08:11:38.799625+0000","last_became_active":"2024-09-11T08:11:31.416491+0000","last_became_peered":"2024-09-11T08:11:31.416491+0000","last_unstale":"2024-09-11T08:11:38.799625+0000","last_undegraded":"2024-09-11T08:11:38.799625+0000","last_fullsized":"2024-09-11T08:11:38.799625+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:38:00.752222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":44,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349976+0000","last_change":"2024-09-11T08:11:27.099368+0000","last_active":"2024-09-11T08:11:39.349976+0000","last_peered":"2024-09-11T08:11:39.349976+0000","last_clean":"2024-09-11T08:11:39.349976+0000","last_became_active":"2024-09-11T08:11:27.098777+0000","last_became_peered":"2024-09-11T08:11:27.098777+0000","last_unstale":"2024-09-11T08:11:39.349976+0000","last_undegraded":"2024-09-11T08:11:39.349976+0000","last_fullsized":"2024-09-11T08:11:39.349976+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:31.994742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"60'1","reported_seq":25,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.808411+0000","last_change":"2024-09-11T08:11:25.088734+0000","last_active":"2024-09-11T08:11:38.808411+0000","last_peered":"2024-09-11T08:11:38.808411+0000","last_clean":"2024-09-11T08:11:38.808411+0000","last_became_active":"2024-09-11T08:11:25.088439+0000","last_became_peered":"2024-09-11T08:11:25.088439+0000","last_unstale":"2024-09-11T08:11:38.808411+0000","last_undegraded":"2024-09-11T08:11:38.808411+0000","last_fullsized":"2024-09-11T08:11:38.808411+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:10:43.793494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.847913+0000","last_change":"2024-09-11T08:11:29.099441+0000","last_active":"2024-09-11T08:11:38.847913+0000","last_peered":"2024-09-11T08:11:38.847913+0000","last_clean":"2024-09-11T08:11:38.847913+0000","last_became_active":"2024-09-11T08:11:29.099164+0000","last_became_peered":"2024-09-11T08:11:29.099164+0000","last_unstale":"2024-09-11T08:11:38.847913+0000","last_undegraded":"2024-09-11T08:11:38.847913+0000","last_fullsized":"2024-09-11T08:11:38.847913+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:44:03.366079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348287+0000","last_change":"2024-09-11T08:11:31.111478+0000","last_active":"2024-09-11T08:11:39.348287+0000","last_peered":"2024-09-11T08:11:39.348287+0000","last_clean":"2024-09-11T08:11:39.348287+0000","last_became_active":"2024-09-11T08:11:31.111172+0000","last_became_peered":"2024-09-11T08:11:31.111172+0000","last_unstale":"2024-09-11T08:11:39.348287+0000","last_undegraded":"2024-09-11T08:11:39.348287+0000","last_fullsized":"2024-09-11T08:11:39.348287+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:19.616502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":31,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349011+0000","last_change":"2024-09-11T08:11:27.097114+0000","last_active":"2024-09-11T08:11:39.349011+0000","last_peered":"2024-09-11T08:11:39.349011+0000","last_clean":"2024-09-11T08:11:39.349011+0000","last_became_active":"2024-09-11T08:11:27.096791+0000","last_became_peered":"2024-09-11T08:11:27.096791+0000","last_unstale":"2024-09-11T08:11:39.349011+0000","last_undegraded":"2024-09-11T08:11:39.349011+0000","last_fullsized":"2024-09-11T08:11:39.349011+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:58.714797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799591+0000","last_change":"2024-09-11T08:11:25.095622+0000","last_active":"2024-09-11T08:11:38.799591+0000","last_peered":"2024-09-11T08:11:38.799591+0000","last_clean":"2024-09-11T08:11:38.799591+0000","last_became_active":"2024-09-11T08:11:25.095374+0000","last_became_peered":"2024-09-11T08:11:25.095374+0000","last_unstale":"2024-09-11T08:11:38.799591+0000","last_undegraded":"2024-09-11T08:11:38.799591+0000","last_fullsized":"2024-09-11T08:11:38.799591+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:49.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":41,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:38.407344+0000","last_change":"2024-09-11T08:11:29.097376+0000","last_active":"2024-09-11T08:12:38.407344+0000","last_peered":"2024-09-11T08:12:38.407344+0000","last_clean":"2024-09-11T08:12:38.407344+0000","last_became_active":"2024-09-11T08:11:29.097072+0000","last_became_peered":"2024-09-11T08:11:29.097072+0000","last_unstale":"2024-09-11T08:12:38.407344+0000","last_undegraded":"2024-09-11T08:12:38.407344+0000","last_fullsized":"2024-09-11T08:12:38.407344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:37:17.423880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349308+0000","last_change":"2024-09-11T08:11:31.112304+0000","last_active":"2024-09-11T08:11:39.349308+0000","last_peered":"2024-09-11T08:11:39.349308+0000","last_clean":"2024-09-11T08:11:39.349308+0000","last_became_active":"2024-09-11T08:11:31.112003+0000","last_became_peered":"2024-09-11T08:11:31.112003+0000","last_unstale":"2024-09-11T08:11:39.349308+0000","last_undegraded":"2024-09-11T08:11:39.349308+0000","last_fullsized":"2024-09-11T08:11:39.349308+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:20:31.627281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"67'9","reported_seq":32,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799680+0000","last_change":"2024-09-11T08:11:27.090518+0000","last_active":"2024-09-11T08:11:38.799680+0000","last_peered":"2024-09-11T08:11:38.799680+0000","last_clean":"2024-09-11T08:11:38.799680+0000","last_became_active":"2024-09-11T08:11:27.090227+0000","last_became_peered":"2024-09-11T08:11:27.090227+0000","last_unstale":"2024-09-11T08:11:38.799680+0000","last_undegraded":"2024-09-11T08:11:38.799680+0000","last_fullsized":"2024-09-11T08:11:38.799680+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:13.167762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798293+0000","last_change":"2024-09-11T08:11:25.093590+0000","last_active":"2024-09-11T08:11:38.798293+0000","last_peered":"2024-09-11T08:11:38.798293+0000","last_clean":"2024-09-11T08:11:38.798293+0000","last_became_active":"2024-09-11T08:11:25.093285+0000","last_became_peered":"2024-09-11T08:11:25.093285+0000","last_unstale":"2024-09-11T08:11:38.798293+0000","last_undegraded":"2024-09-11T08:11:38.798293+0000","last_fullsized":"2024-09-11T08:11:38.798293+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:06:09.251490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348593+0000","last_change":"2024-09-11T08:11:29.092048+0000","last_active":"2024-09-11T08:11:39.348593+0000","last_peered":"2024-09-11T08:11:39.348593+0000","last_clean":"2024-09-11T08:11:39.348593+0000","last_became_active":"2024-09-11T08:11:29.091748+0000","last_became_peered":"2024-09-11T08:11:29.091748+0000","last_unstale":"2024-09-11T08:11:39.348593+0000","last_undegraded":"2024-09-11T08:11:39.348593+0000","last_fullsized":"2024-09-11T08:11:39.348593+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:59:59.797833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798653+0000","last_change":"2024-09-11T08:11:31.114435+0000","last_active":"2024-09-11T08:11:38.798653+0000","last_peered":"2024-09-11T08:11:38.798653+0000","last_clean":"2024-09-11T08:11:38.798653+0000","last_became_active":"2024-09-11T08:11:31.114063+0000","last_became_peered":"2024-09-11T08:11:31.114063+0000","last_unstale":"2024-09-11T08:11:38.798653+0000","last_undegraded":"2024-09-11T08:11:38.798653+0000","last_fullsized":"2024-09-11T08:11:38.798653+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:08:59.278736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"67'19","reported_seq":47,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.808320+0000","last_change":"2024-09-11T08:11:27.089754+0000","last_active":"2024-09-11T08:11:38.808320+0000","last_peered":"2024-09-11T08:11:38.808320+0000","last_clean":"2024-09-11T08:11:38.808320+0000","last_became_active":"2024-09-11T08:11:27.089090+0000","last_became_peered":"2024-09-11T08:11:27.089090+0000","last_unstale":"2024-09-11T08:11:38.808320+0000","last_undegraded":"2024-09-11T08:11:38.808320+0000","last_fullsized":"2024-09-11T08:11:38.808320+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:18:32.773640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349374+0000","last_change":"2024-09-11T08:11:25.090325+0000","last_active":"2024-09-11T08:11:39.349374+0000","last_peered":"2024-09-11T08:11:39.349374+0000","last_clean":"2024-09-11T08:11:39.349374+0000","last_became_active":"2024-09-11T08:11:25.090057+0000","last_became_peered":"2024-09-11T08:11:25.090057+0000","last_unstale":"2024-09-11T08:11:39.349374+0000","last_undegraded":"2024-09-11T08:11:39.349374+0000","last_fullsized":"2024-09-11T08:11:39.349374+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:57.545046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348496+0000","last_change":"2024-09-11T08:11:29.099277+0000","last_active":"2024-09-11T08:11:39.348496+0000","last_peered":"2024-09-11T08:11:39.348496+0000","last_clean":"2024-09-11T08:11:39.348496+0000","last_became_active":"2024-09-11T08:11:29.098960+0000","last_became_peered":"2024-09-11T08:11:29.098960+0000","last_unstale":"2024-09-11T08:11:39.348496+0000","last_undegraded":"2024-09-11T08:11:39.348496+0000","last_fullsized":"2024-09-11T08:11:39.348496+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:40:53.810999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348485+0000","last_change":"2024-09-11T08:11:31.105821+0000","last_active":"2024-09-11T08:11:39.348485+0000","last_peered":"2024-09-11T08:11:39.348485+0000","last_clean":"2024-09-11T08:11:39.348485+0000","last_became_active":"2024-09-11T08:11:31.105542+0000","last_became_peered":"2024-09-11T08:11:31.105542+0000","last_unstale":"2024-09-11T08:11:39.348485+0000","last_undegraded":"2024-09-11T08:11:39.348485+0000","last_fullsized":"2024-09-11T08:11:39.348485+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:06:17.897009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"67'12","reported_seq":39,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349960+0000","last_change":"2024-09-11T08:11:27.099017+0000","last_active":"2024-09-11T08:11:39.349960+0000","last_peered":"2024-09-11T08:11:39.349960+0000","last_clean":"2024-09-11T08:11:39.349960+0000","last_became_active":"2024-09-11T08:11:27.098583+0000","last_became_peered":"2024-09-11T08:11:27.098583+0000","last_unstale":"2024-09-11T08:11:39.349960+0000","last_undegraded":"2024-09-11T08:11:39.349960+0000","last_fullsized":"2024-09-11T08:11:39.349960+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:44:05.894625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349850+0000","last_change":"2024-09-11T08:11:25.088852+0000","last_active":"2024-09-11T08:11:39.349850+0000","last_peered":"2024-09-11T08:11:39.349850+0000","last_clean":"2024-09-11T08:11:39.349850+0000","last_became_active":"2024-09-11T08:11:25.088519+0000","last_became_peered":"2024-09-11T08:11:25.088519+0000","last_unstale":"2024-09-11T08:11:39.349850+0000","last_undegraded":"2024-09-11T08:11:39.349850+0000","last_fullsized":"2024-09-11T08:11:39.349850+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:14:17.016711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348656+0000","last_change":"2024-09-11T08:11:29.099409+0000","last_active":"2024-09-11T08:11:39.348656+0000","last_peered":"2024-09-11T08:11:39.348656+0000","last_clean":"2024-09-11T08:11:39.348656+0000","last_became_active":"2024-09-11T08:11:29.099131+0000","last_became_peered":"2024-09-11T08:11:29.099131+0000","last_unstale":"2024-09-11T08:11:39.348656+0000","last_undegraded":"2024-09-11T08:11:39.348656+0000","last_fullsized":"2024-09-11T08:11:39.348656+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:56:11.067512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799291+0000","last_change":"2024-09-11T08:11:31.110912+0000","last_active":"2024-09-11T08:11:38.799291+0000","last_peered":"2024-09-11T08:11:38.799291+0000","last_clean":"2024-09-11T08:11:38.799291+0000","last_became_active":"2024-09-11T08:11:31.110648+0000","last_became_peered":"2024-09-11T08:11:31.110648+0000","last_unstale":"2024-09-11T08:11:38.799291+0000","last_undegraded":"2024-09-11T08:11:38.799291+0000","last_fullsized":"2024-09-11T08:11:38.799291+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:55.345946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"67'15","reported_seq":41,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.797916+0000","last_change":"2024-09-11T08:11:27.086366+0000","last_active":"2024-09-11T08:11:38.797916+0000","last_peered":"2024-09-11T08:11:38.797916+0000","last_clean":"2024-09-11T08:11:38.797916+0000","last_became_active":"2024-09-11T08:11:27.086081+0000","last_became_peered":"2024-09-11T08:11:27.086081+0000","last_unstale":"2024-09-11T08:11:38.797916+0000","last_undegraded":"2024-09-11T08:11:38.797916+0000","last_fullsized":"2024-09-11T08:11:38.797916+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:53:26.600789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"60'2","reported_seq":35,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349654+0000","last_change":"2024-09-11T08:11:25.094547+0000","last_active":"2024-09-11T08:11:39.349654+0000","last_peered":"2024-09-11T08:11:39.349654+0000","last_clean":"2024-09-11T08:11:39.349654+0000","last_became_active":"2024-09-11T08:11:25.094277+0000","last_became_peered":"2024-09-11T08:11:25.094277+0000","last_unstale":"2024-09-11T08:11:39.349654+0000","last_undegraded":"2024-09-11T08:11:39.349654+0000","last_fullsized":"2024-09-11T08:11:39.349654+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:02:20.460779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":38,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:38.407438+0000","last_change":"2024-09-11T08:11:29.097168+0000","last_active":"2024-09-11T08:12:38.407438+0000","last_peered":"2024-09-11T08:12:38.407438+0000","last_clean":"2024-09-11T08:12:38.407438+0000","last_became_active":"2024-09-11T08:11:29.096734+0000","last_became_peered":"2024-09-11T08:11:29.096734+0000","last_unstale":"2024-09-11T08:12:38.407438+0000","last_undegraded":"2024-09-11T08:12:38.407438+0000","last_fullsized":"2024-09-11T08:12:38.407438+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:05:24.289993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798263+0000","last_change":"2024-09-11T08:11:31.418399+0000","last_active":"2024-09-11T08:11:38.798263+0000","last_peered":"2024-09-11T08:11:38.798263+0000","last_clean":"2024-09-11T08:11:38.798263+0000","last_became_active":"2024-09-11T08:11:31.418101+0000","last_became_peered":"2024-09-11T08:11:31.418101+0000","last_unstale":"2024-09-11T08:11:38.798263+0000","last_undegraded":"2024-09-11T08:11:38.798263+0000","last_fullsized":"2024-09-11T08:11:38.798263+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:01:10.314168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.808196+0000","last_change":"2024-09-11T08:11:25.094098+0000","last_active":"2024-09-11T08:11:38.808196+0000","last_peered":"2024-09-11T08:11:38.808196+0000","last_clean":"2024-09-11T08:11:38.808196+0000","last_became_active":"2024-09-11T08:11:25.093678+0000","last_became_peered":"2024-09-11T08:11:25.093678+0000","last_unstale":"2024-09-11T08:11:38.808196+0000","last_undegraded":"2024-09-11T08:11:38.808196+0000","last_fullsized":"2024-09-11T08:11:38.808196+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:42:39.518185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"67'6","reported_seq":25,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.800418+0000","last_change":"2024-09-11T08:11:27.090858+0000","last_active":"2024-09-11T08:11:38.800418+0000","last_peered":"2024-09-11T08:11:38.800418+0000","last_clean":"2024-09-11T08:11:38.800418+0000","last_became_active":"2024-09-11T08:11:27.090545+0000","last_became_peered":"2024-09-11T08:11:27.090545+0000","last_unstale":"2024-09-11T08:11:38.800418+0000","last_undegraded":"2024-09-11T08:11:38.800418+0000","last_fullsized":"2024-09-11T08:11:38.800418+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:41.233522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798344+0000","last_change":"2024-09-11T08:11:29.092988+0000","last_active":"2024-09-11T08:11:38.798344+0000","last_peered":"2024-09-11T08:11:38.798344+0000","last_clean":"2024-09-11T08:11:38.798344+0000","last_became_active":"2024-09-11T08:11:29.092698+0000","last_became_peered":"2024-09-11T08:11:29.092698+0000","last_unstale":"2024-09-11T08:11:38.798344+0000","last_undegraded":"2024-09-11T08:11:38.798344+0000","last_fullsized":"2024-09-11T08:11:38.798344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:37.407747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348994+0000","last_change":"2024-09-11T08:11:31.418227+0000","last_active":"2024-09-11T08:11:39.348994+0000","last_peered":"2024-09-11T08:11:39.348994+0000","last_clean":"2024-09-11T08:11:39.348994+0000","last_became_active":"2024-09-11T08:11:31.417918+0000","last_became_peered":"2024-09-11T08:11:31.417918+0000","last_unstale":"2024-09-11T08:11:39.348994+0000","last_undegraded":"2024-09-11T08:11:39.348994+0000","last_fullsized":"2024-09-11T08:11:39.348994+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:24:23.646738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"67'9","reported_seq":32,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799758+0000","last_change":"2024-09-11T08:11:27.093531+0000","last_active":"2024-09-11T08:11:38.799758+0000","last_peered":"2024-09-11T08:11:38.799758+0000","last_clean":"2024-09-11T08:11:38.799758+0000","last_became_active":"2024-09-11T08:11:27.093241+0000","last_became_peered":"2024-09-11T08:11:27.093241+0000","last_unstale":"2024-09-11T08:11:38.799758+0000","last_undegraded":"2024-09-11T08:11:38.799758+0000","last_fullsized":"2024-09-11T08:11:38.799758+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:58:48.283877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349188+0000","last_change":"2024-09-11T08:11:25.090713+0000","last_active":"2024-09-11T08:11:39.349188+0000","last_peered":"2024-09-11T08:11:39.349188+0000","last_clean":"2024-09-11T08:11:39.349188+0000","last_became_active":"2024-09-11T08:11:25.090416+0000","last_became_peered":"2024-09-11T08:11:25.090416+0000","last_unstale":"2024-09-11T08:11:39.349188+0000","last_undegraded":"2024-09-11T08:11:39.349188+0000","last_fullsized":"2024-09-11T08:11:39.349188+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:15.760643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.800273+0000","last_change":"2024-09-11T08:11:29.096594+0000","last_active":"2024-09-11T08:11:38.800273+0000","last_peered":"2024-09-11T08:11:38.800273+0000","last_clean":"2024-09-11T08:11:38.800273+0000","last_became_active":"2024-09-11T08:11:29.096267+0000","last_became_peered":"2024-09-11T08:11:29.096267+0000","last_unstale":"2024-09-11T08:11:38.800273+0000","last_undegraded":"2024-09-11T08:11:38.800273+0000","last_fullsized":"2024-09-11T08:11:38.800273+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:04:00.659569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349069+0000","last_change":"2024-09-11T08:11:31.107301+0000","last_active":"2024-09-11T08:11:39.349069+0000","last_peered":"2024-09-11T08:11:39.349069+0000","last_clean":"2024-09-11T08:11:39.349069+0000","last_became_active":"2024-09-11T08:11:31.106979+0000","last_became_peered":"2024-09-11T08:11:31.106979+0000","last_unstale":"2024-09-11T08:11:39.349069+0000","last_undegraded":"2024-09-11T08:11:39.349069+0000","last_fullsized":"2024-09-11T08:11:39.349069+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:32:14.830059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"67'9","reported_seq":32,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798542+0000","last_change":"2024-09-11T08:11:27.085114+0000","last_active":"2024-09-11T08:11:38.798542+0000","last_peered":"2024-09-11T08:11:38.798542+0000","last_clean":"2024-09-11T08:11:38.798542+0000","last_became_active":"2024-09-11T08:11:27.084820+0000","last_became_peered":"2024-09-11T08:11:27.084820+0000","last_unstale":"2024-09-11T08:11:38.798542+0000","last_undegraded":"2024-09-11T08:11:38.798542+0000","last_fullsized":"2024-09-11T08:11:38.798542+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:23.041169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799108+0000","last_change":"2024-09-11T08:11:25.092328+0000","last_active":"2024-09-11T08:11:38.799108+0000","last_peered":"2024-09-11T08:11:38.799108+0000","last_clean":"2024-09-11T08:11:38.799108+0000","last_became_active":"2024-09-11T08:11:25.091878+0000","last_became_peered":"2024-09-11T08:11:25.091878+0000","last_unstale":"2024-09-11T08:11:38.799108+0000","last_undegraded":"2024-09-11T08:11:38.799108+0000","last_fullsized":"2024-09-11T08:11:38.799108+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:02.395988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":38,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:38.407364+0000","last_change":"2024-09-11T08:11:29.091371+0000","last_active":"2024-09-11T08:12:38.407364+0000","last_peered":"2024-09-11T08:12:38.407364+0000","last_clean":"2024-09-11T08:12:38.407364+0000","last_became_active":"2024-09-11T08:11:29.091021+0000","last_became_peered":"2024-09-11T08:11:29.091021+0000","last_unstale":"2024-09-11T08:12:38.407364+0000","last_undegraded":"2024-09-11T08:12:38.407364+0000","last_fullsized":"2024-09-11T08:12:38.407364+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:03:11.692469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348602+0000","last_change":"2024-09-11T08:11:31.105834+0000","last_active":"2024-09-11T08:11:39.348602+0000","last_peered":"2024-09-11T08:11:39.348602+0000","last_clean":"2024-09-11T08:11:39.348602+0000","last_became_active":"2024-09-11T08:11:31.105487+0000","last_became_peered":"2024-09-11T08:11:31.105487+0000","last_unstale":"2024-09-11T08:11:39.348602+0000","last_undegraded":"2024-09-11T08:11:39.348602+0000","last_fullsized":"2024-09-11T08:11:39.348602+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:49:24.252947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"67'10","reported_seq":31,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.800049+0000","last_change":"2024-09-11T08:11:27.087770+0000","last_active":"2024-09-11T08:11:38.800049+0000","last_peered":"2024-09-11T08:11:38.800049+0000","last_clean":"2024-09-11T08:11:38.800049+0000","last_became_active":"2024-09-11T08:11:27.086652+0000","last_became_peered":"2024-09-11T08:11:27.086652+0000","last_unstale":"2024-09-11T08:11:38.800049+0000","last_undegraded":"2024-09-11T08:11:38.800049+0000","last_fullsized":"2024-09-11T08:11:38.800049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:39:59.515088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349248+0000","last_change":"2024-09-11T08:11:25.090560+0000","last_active":"2024-09-11T08:11:39.349248+0000","last_peered":"2024-09-11T08:11:39.349248+0000","last_clean":"2024-09-11T08:11:39.349248+0000","last_became_active":"2024-09-11T08:11:25.090195+0000","last_became_peered":"2024-09-11T08:11:25.090195+0000","last_unstale":"2024-09-11T08:11:39.349248+0000","last_undegraded":"2024-09-11T08:11:39.349248+0000","last_fullsized":"2024-09-11T08:11:39.349248+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:52.830520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":38,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:38.407438+0000","last_change":"2024-09-11T08:11:29.099539+0000","last_active":"2024-09-11T08:12:38.407438+0000","last_peered":"2024-09-11T08:12:38.407438+0000","last_clean":"2024-09-11T08:12:38.407438+0000","last_became_active":"2024-09-11T08:11:29.099258+0000","last_became_peered":"2024-09-11T08:11:29.099258+0000","last_unstale":"2024-09-11T08:12:38.407438+0000","last_undegraded":"2024-09-11T08:12:38.407438+0000","last_fullsized":"2024-09-11T08:12:38.407438+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:57:12.119930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799362+0000","last_change":"2024-09-11T08:11:31.113611+0000","last_active":"2024-09-11T08:11:38.799362+0000","last_peered":"2024-09-11T08:11:38.799362+0000","last_clean":"2024-09-11T08:11:38.799362+0000","last_became_active":"2024-09-11T08:11:31.113314+0000","last_became_peered":"2024-09-11T08:11:31.113314+0000","last_unstale":"2024-09-11T08:11:38.799362+0000","last_undegraded":"2024-09-11T08:11:38.799362+0000","last_fullsized":"2024-09-11T08:11:38.799362+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:42:47.265588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":35,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349016+0000","last_change":"2024-09-11T08:11:27.099097+0000","last_active":"2024-09-11T08:11:39.349016+0000","last_peered":"2024-09-11T08:11:39.349016+0000","last_clean":"2024-09-11T08:11:39.349016+0000","last_became_active":"2024-09-11T08:11:27.098659+0000","last_became_peered":"2024-09-11T08:11:27.098659+0000","last_unstale":"2024-09-11T08:11:39.349016+0000","last_undegraded":"2024-09-11T08:11:39.349016+0000","last_fullsized":"2024-09-11T08:11:39.349016+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:05:53.276172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349190+0000","last_change":"2024-09-11T08:11:25.091639+0000","last_active":"2024-09-11T08:11:39.349190+0000","last_peered":"2024-09-11T08:11:39.349190+0000","last_clean":"2024-09-11T08:11:39.349190+0000","last_became_active":"2024-09-11T08:11:25.091358+0000","last_became_peered":"2024-09-11T08:11:25.091358+0000","last_unstale":"2024-09-11T08:11:39.349190+0000","last_undegraded":"2024-09-11T08:11:39.349190+0000","last_fullsized":"2024-09-11T08:11:39.349190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:44.266849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.847978+0000","last_change":"2024-09-11T08:11:29.094531+0000","last_active":"2024-09-11T08:11:38.847978+0000","last_peered":"2024-09-11T08:11:38.847978+0000","last_clean":"2024-09-11T08:11:38.847978+0000","last_became_active":"2024-09-11T08:11:29.094244+0000","last_became_peered":"2024-09-11T08:11:29.094244+0000","last_unstale":"2024-09-11T08:11:38.847978+0000","last_undegraded":"2024-09-11T08:11:38.847978+0000","last_fullsized":"2024-09-11T08:11:38.847978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:35.475093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799518+0000","last_change":"2024-09-11T08:11:31.110223+0000","last_active":"2024-09-11T08:11:38.799518+0000","last_peered":"2024-09-11T08:11:38.799518+0000","last_clean":"2024-09-11T08:11:38.799518+0000","last_became_active":"2024-09-11T08:11:31.109929+0000","last_became_peered":"2024-09-11T08:11:31.109929+0000","last_unstale":"2024-09-11T08:11:38.799518+0000","last_undegraded":"2024-09-11T08:11:38.799518+0000","last_fullsized":"2024-09-11T08:11:38.799518+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:46:50.447850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":32,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.848206+0000","last_change":"2024-09-11T08:11:27.095479+0000","last_active":"2024-09-11T08:11:38.848206+0000","last_peered":"2024-09-11T08:11:38.848206+0000","last_clean":"2024-09-11T08:11:38.848206+0000","last_became_active":"2024-09-11T08:11:27.091860+0000","last_became_peered":"2024-09-11T08:11:27.091860+0000","last_unstale":"2024-09-11T08:11:38.848206+0000","last_undegraded":"2024-09-11T08:11:38.848206+0000","last_fullsized":"2024-09-11T08:11:38.848206+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:14.627903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349254+0000","last_change":"2024-09-11T08:11:25.089770+0000","last_active":"2024-09-11T08:11:39.349254+0000","last_peered":"2024-09-11T08:11:39.349254+0000","last_clean":"2024-09-11T08:11:39.349254+0000","last_became_active":"2024-09-11T08:11:25.089471+0000","last_became_peered":"2024-09-11T08:11:25.089471+0000","last_unstale":"2024-09-11T08:11:39.349254+0000","last_undegraded":"2024-09-11T08:11:39.349254+0000","last_fullsized":"2024-09-11T08:11:39.349254+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:10.414780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799364+0000","last_change":"2024-09-11T08:11:29.099521+0000","last_active":"2024-09-11T08:11:38.799364+0000","last_peered":"2024-09-11T08:11:38.799364+0000","last_clean":"2024-09-11T08:11:38.799364+0000","last_became_active":"2024-09-11T08:11:29.099072+0000","last_became_peered":"2024-09-11T08:11:29.099072+0000","last_unstale":"2024-09-11T08:11:38.799364+0000","last_undegraded":"2024-09-11T08:11:38.799364+0000","last_fullsized":"2024-09-11T08:11:38.799364+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:43.207670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798370+0000","last_change":"2024-09-11T08:11:31.108660+0000","last_active":"2024-09-11T08:11:38.798370+0000","last_peered":"2024-09-11T08:11:38.798370+0000","last_clean":"2024-09-11T08:11:38.798370+0000","last_became_active":"2024-09-11T08:11:31.108359+0000","last_became_peered":"2024-09-11T08:11:31.108359+0000","last_unstale":"2024-09-11T08:11:38.798370+0000","last_undegraded":"2024-09-11T08:11:38.798370+0000","last_fullsized":"2024-09-11T08:11:38.798370+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:34.391986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"67'11","reported_seq":35,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798637+0000","last_change":"2024-09-11T08:11:27.085607+0000","last_active":"2024-09-11T08:11:38.798637+0000","last_peered":"2024-09-11T08:11:38.798637+0000","last_clean":"2024-09-11T08:11:38.798637+0000","last_became_active":"2024-09-11T08:11:27.085326+0000","last_became_peered":"2024-09-11T08:11:27.085326+0000","last_unstale":"2024-09-11T08:11:38.798637+0000","last_undegraded":"2024-09-11T08:11:38.798637+0000","last_fullsized":"2024-09-11T08:11:38.798637+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:32:39.314248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798380+0000","last_change":"2024-09-11T08:11:25.097786+0000","last_active":"2024-09-11T08:11:38.798380+0000","last_peered":"2024-09-11T08:11:38.798380+0000","last_clean":"2024-09-11T08:11:38.798380+0000","last_became_active":"2024-09-11T08:11:25.097485+0000","last_became_peered":"2024-09-11T08:11:25.097485+0000","last_unstale":"2024-09-11T08:11:38.798380+0000","last_undegraded":"2024-09-11T08:11:38.798380+0000","last_fullsized":"2024-09-11T08:11:38.798380+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:28.843596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348509+0000","last_change":"2024-09-11T08:11:29.097789+0000","last_active":"2024-09-11T08:11:39.348509+0000","last_peered":"2024-09-11T08:11:39.348509+0000","last_clean":"2024-09-11T08:11:39.348509+0000","last_became_active":"2024-09-11T08:11:29.097518+0000","last_became_peered":"2024-09-11T08:11:29.097518+0000","last_unstale":"2024-09-11T08:11:39.348509+0000","last_undegraded":"2024-09-11T08:11:39.348509+0000","last_fullsized":"2024-09-11T08:11:39.348509+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:44.929779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799522+0000","last_change":"2024-09-11T08:11:31.417458+0000","last_active":"2024-09-11T08:11:38.799522+0000","last_peered":"2024-09-11T08:11:38.799522+0000","last_clean":"2024-09-11T08:11:38.799522+0000","last_became_active":"2024-09-11T08:11:31.416299+0000","last_became_peered":"2024-09-11T08:11:31.416299+0000","last_unstale":"2024-09-11T08:11:38.799522+0000","last_undegraded":"2024-09-11T08:11:38.799522+0000","last_fullsized":"2024-09-11T08:11:38.799522+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:13.575094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":22,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799105+0000","last_change":"2024-09-11T08:11:27.089964+0000","last_active":"2024-09-11T08:11:38.799105+0000","last_peered":"2024-09-11T08:11:38.799105+0000","last_clean":"2024-09-11T08:11:38.799105+0000","last_became_active":"2024-09-11T08:11:27.086636+0000","last_became_peered":"2024-09-11T08:11:27.086636+0000","last_unstale":"2024-09-11T08:11:38.799105+0000","last_undegraded":"2024-09-11T08:11:38.799105+0000","last_fullsized":"2024-09-11T08:11:38.799105+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:32.772749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799178+0000","last_change":"2024-09-11T08:11:25.100896+0000","last_active":"2024-09-11T08:11:38.799178+0000","last_peered":"2024-09-11T08:11:38.799178+0000","last_clean":"2024-09-11T08:11:38.799178+0000","last_became_active":"2024-09-11T08:11:25.100298+0000","last_became_peered":"2024-09-11T08:11:25.100298+0000","last_unstale":"2024-09-11T08:11:38.799178+0000","last_undegraded":"2024-09-11T08:11:38.799178+0000","last_fullsized":"2024-09-11T08:11:38.799178+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:57:41.021067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.808217+0000","last_change":"2024-09-11T08:11:29.098400+0000","last_active":"2024-09-11T08:11:38.808217+0000","last_peered":"2024-09-11T08:11:38.808217+0000","last_clean":"2024-09-11T08:11:38.808217+0000","last_became_active":"2024-09-11T08:11:29.098104+0000","last_became_peered":"2024-09-11T08:11:29.098104+0000","last_unstale":"2024-09-11T08:11:38.808217+0000","last_undegraded":"2024-09-11T08:11:38.808217+0000","last_fullsized":"2024-09-11T08:11:38.808217+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:08:10.525216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348612+0000","last_change":"2024-09-11T08:11:31.109269+0000","last_active":"2024-09-11T08:11:39.348612+0000","last_peered":"2024-09-11T08:11:39.348612+0000","last_clean":"2024-09-11T08:11:39.348612+0000","last_became_active":"2024-09-11T08:11:31.108848+0000","last_became_peered":"2024-09-11T08:11:31.108848+0000","last_unstale":"2024-09-11T08:11:39.348612+0000","last_undegraded":"2024-09-11T08:11:39.348612+0000","last_fullsized":"2024-09-11T08:11:39.348612+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:17.843343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.848093+0000","last_change":"2024-09-11T08:11:31.104544+0000","last_active":"2024-09-11T08:11:38.848093+0000","last_peered":"2024-09-11T08:11:38.848093+0000","last_clean":"2024-09-11T08:11:38.848093+0000","last_became_active":"2024-09-11T08:11:31.104169+0000","last_became_peered":"2024-09-11T08:11:31.104169+0000","last_unstale":"2024-09-11T08:11:38.848093+0000","last_undegraded":"2024-09-11T08:11:38.848093+0000","last_fullsized":"2024-09-11T08:11:38.848093+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:02.802258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799322+0000","last_change":"2024-09-11T08:11:25.096405+0000","last_active":"2024-09-11T08:11:38.799322+0000","last_peered":"2024-09-11T08:11:38.799322+0000","last_clean":"2024-09-11T08:11:38.799322+0000","last_became_active":"2024-09-11T08:11:25.095993+0000","last_became_peered":"2024-09-11T08:11:25.095993+0000","last_unstale":"2024-09-11T08:11:38.799322+0000","last_undegraded":"2024-09-11T08:11:38.799322+0000","last_fullsized":"2024-09-11T08:11:38.799322+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:46.111533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"67'11","reported_seq":35,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.808616+0000","last_change":"2024-09-11T08:11:27.089123+0000","last_active":"2024-09-11T08:11:38.808616+0000","last_peered":"2024-09-11T08:11:38.808616+0000","last_clean":"2024-09-11T08:11:38.808616+0000","last_became_active":"2024-09-11T08:11:27.088855+0000","last_became_peered":"2024-09-11T08:11:27.088855+0000","last_unstale":"2024-09-11T08:11:38.808616+0000","last_undegraded":"2024-09-11T08:11:38.808616+0000","last_fullsized":"2024-09-11T08:11:38.808616+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:14:01.493971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799435+0000","last_change":"2024-09-11T08:11:29.100530+0000","last_active":"2024-09-11T08:11:38.799435+0000","last_peered":"2024-09-11T08:11:38.799435+0000","last_clean":"2024-09-11T08:11:38.799435+0000","last_became_active":"2024-09-11T08:11:29.100218+0000","last_became_peered":"2024-09-11T08:11:29.100218+0000","last_unstale":"2024-09-11T08:11:38.799435+0000","last_undegraded":"2024-09-11T08:11:38.799435+0000","last_fullsized":"2024-09-11T08:11:38.799435+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:25.054678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":258,"num_write":179,"num_write_kb":3194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234006,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298688,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442368,"statfs":{"total":95990841344,"available":95684984832,"internally_reserved":0,"allocated":1363968,"data_stored":907109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.49299999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.437}]},{"osd":2,"last update":"Wed Sep 11 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.44600000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.46700000000000003}]},{"osd":4,"last update":"Wed Sep 11 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.41699999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.34100000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.38700000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495643,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298660,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442396,"statfs":{"total":95990841344,"available":95685013504,"internally_reserved":0,"allocated":1335296,"data_stored":905819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:11:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.433},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.497}]},{"osd":1,"last update":"Wed Sep 11 08:11:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.45100000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:11:48 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.38},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:11:48 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.441},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.48299999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:11:48 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.40899999999999997}]},{"osd":5,"last update":"Wed Sep 11 08:11:48 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.47199999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.61899999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757278,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298092,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442964,"statfs":{"total":95990841344,"available":95685595136,"internally_reserved":0,"allocated":753664,"data_stored":316015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:12:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.52300000000000002,"15min":0.53000000000000003},"min":{"1min":0.34300000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.69899999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47099999999999997}]},{"osd":1,"last update":"Wed Sep 11 08:12:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52700000000000002,"15min":0.52800000000000002},"min":{"1min":0.34599999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.73799999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.55100000000000005,"15min":0.56299999999999994},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.745,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.54300000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:12:30 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.54000000000000004,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.501},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.39900000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66000000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.44700000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:12:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.33500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68400000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.47899999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:12:30 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.67200000000000004,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.309,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.51600000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.36699999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:11:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.41699999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:12:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.51900000000000002}]}]},{"osd":4,"up_from":30,"seq":128849018916,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298108,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442948,"statfs":{"total":95990841344,"available":95685578752,"internally_reserved":0,"allocated":770048,"data_stored":317103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:12:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.51500000000000001,"15min":0.50800000000000001},"min":{"1min":0.42599999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.39200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.498}]},{"osd":1,"last update":"Wed Sep 11 08:12:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.438},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.54400000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:12:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.42299999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.44800000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:12:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52300000000000002,"15min":0.51800000000000002},"min":{"1min":0.41699999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.47999999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:12:26 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.375,"15min":0.36899999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.375,"15min":0.36599999999999999},"min":{"1min":0.31900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.32900000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.318},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.38}]},{"osd":7,"last update":"Wed Sep 11 08:12:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.36499999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182441,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298116,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442940,"statfs":{"total":95990841344,"available":95685570560,"internally_reserved":0,"allocated":778240,"data_stored":316522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:12:45 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.34100000000000003,"15min":0.34499999999999997},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.41999999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.375},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.47799999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.437}]},{"osd":1,"last update":"Wed Sep 11 08:12:45 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.376,"15min":0.38},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.51500000000000001,"5min":0.626,"15min":0.626},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.40799999999999997,"15min":0.42799999999999999},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48099999999999998,"5min":0.751,"15min":0.751},"last":0.36199999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:12:45 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40999999999999998,"15min":0.41999999999999998},"min":{"1min":0.29399999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.49399999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.39300000000000002,"15min":0.40500000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.46600000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.44700000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:12:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52800000000000002,"15min":0.53200000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.61899999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:12:27 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.53500000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56599999999999995}]},{"osd":6,"last update":"Wed Sep 11 08:11:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63}]},{"osd":7,"last update":"Wed Sep 11 08:12:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65500000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298080,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442976,"statfs":{"total":95990841344,"available":95685607424,"internally_reserved":0,"allocated":741376,"data_stored":315143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:12:16 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36199999999999999,"15min":0.35399999999999998},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.60199999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.36499999999999999,"15min":0.36299999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.52300000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.32100000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:12:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.38100000000000001,"15min":0.377},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.54500000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58099999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.35199999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:12:46 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.42899999999999999,"15min":0.434},"min":{"1min":0.28799999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.57699999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.442,"15min":0.44700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63700000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.39300000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:12:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55200000000000005,"15min":0.55600000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69899999999999995,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68799999999999994,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.47899999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:12:30 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71999999999999997,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.47099999999999997}]},{"osd":6,"last update":"Wed Sep 11 08:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.57499999999999996}]},{"osd":7,"last update":"Wed Sep 11 08:12:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.59599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298148,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442908,"statfs":{"total":95990841344,"available":95685537792,"internally_reserved":0,"allocated":811008,"data_stored":318606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.34300000000000003,"15min":0.32300000000000001},"min":{"1min":0.224,"5min":0.221,"15min":0.221},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.34799999999999998},"min":{"1min":0.255,"5min":0.214,"15min":0.214},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.47599999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:12:18 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36099999999999999,"15min":0.35099999999999998},"min":{"1min":0.23599999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.54500000000000004,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40699999999999997,"15min":0.40300000000000002},"min":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.41499999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:12:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.45700000000000002,"15min":0.46300000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.622,"5min":0.747,"15min":0.747},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.65500000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.38300000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:12:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55200000000000005,"15min":0.54300000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.49399999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:12:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.50600000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52600000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:12:13 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.59399999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298684,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442372,"statfs":{"total":95990841344,"available":95684988928,"internally_reserved":0,"allocated":1363968,"data_stored":907667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.32700000000000001,"15min":0.317},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.434,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.316},"min":{"1min":0.22900000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.41799999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:12:22 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.36099999999999999,"15min":0.36499999999999999},"min":{"1min":0.27200000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.373},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.38100000000000001},"min":{"1min":0.26600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.49299999999999999,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.38800000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:12:43 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40600000000000003,"15min":0.41199999999999998},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.55700000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.41799999999999998,"15min":0.42299999999999999},"min":{"1min":0.28899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.498,"5min":0.62,"15min":0.62},"last":0.438}]},{"osd":4,"last update":"Wed Sep 11 08:12:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.502,"15min":0.496},"min":{"1min":0.436,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.65700000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.48099999999999998,"15min":0.48199999999999998},"min":{"1min":0.35999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.51400000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:12:27 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.39600000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.47599999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.58099999999999996}]},{"osd":7,"last update":"Wed Sep 11 08:12:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.502}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:12:52.383 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:12:52.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:12:52 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug there is no tcmu-runner data available 2024-09-11T08:12:52.634 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:52.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:52 smithi071 ceph-mon[25834]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:52.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:52 smithi071 ceph-mon[39897]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:52.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:52 smithi196 ceph-mon[30677]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:53.562 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 bash[72664]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T08:12:53.562 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 bash[72664]: Writing manifest to image destination 2024-09-11T08:12:53.838 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-mon[30677]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:12:53.838 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-mon[30677]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:53.838 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 podman[72664]: 2024-09-11 08:12:53.615229725 +0000 UTC m=+2.246404057 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T08:12:53.838 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 podman[72664]: 2024-09-11 08:12:53.72213546 +0000 UTC m=+2.353309818 container create 214d90a9f27e6e0c8dbf765676b5da39631661328972f08d9bac2677f29c9325 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:12:53.944 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:53 smithi071 ceph-mon[39897]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:12:53.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:53 smithi071 ceph-mon[39897]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:53.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:53 smithi071 ceph-mon[25834]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:12:53.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:53 smithi071 ceph-mon[25834]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:54.181 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 podman[72664]: 2024-09-11 08:12:53.957693727 +0000 UTC m=+2.588868047 container init 214d90a9f27e6e0c8dbf765676b5da39631661328972f08d9bac2677f29c9325 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:12:54.181 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 podman[72664]: 2024-09-11 08:12:53.962731889 +0000 UTC m=+2.593906206 container start 214d90a9f27e6e0c8dbf765676b5da39631661328972f08d9bac2677f29c9325 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:12:54.181 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.966Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T08:12:54.181 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.966Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T08:12:54.181 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.966Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T08:12:54.182 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.966Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T08:12:54.182 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T08:12:54.182 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T08:12:54.182 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T08:12:54.182 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T08:12:54.182 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T08:12:54.182 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T08:12:54.182 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T08:12:54.182 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T08:12:54.182 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T08:12:54.182 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T08:12:54.183 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T08:12:54.183 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T08:12:54.183 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T08:12:54.183 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T08:12:54.183 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T08:12:54.183 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T08:12:54.183 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T08:12:54.183 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T08:12:54.183 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T08:12:54.183 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T08:12:54.184 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T08:12:54.184 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T08:12:54.184 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T08:12:54.184 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T08:12:54.184 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T08:12:54.184 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T08:12:54.184 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T08:12:54.184 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T08:12:54.184 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T08:12:54.184 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T08:12:54.184 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T08:12:54.184 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T08:12:54.184 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T08:12:54.184 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T08:12:54.185 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T08:12:54.185 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T08:12:54.185 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T08:12:54.185 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T08:12:54.185 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T08:12:54.185 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T08:12:54.185 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T08:12:54.185 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T08:12:54.185 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T08:12:54.185 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T08:12:54.185 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T08:12:54.185 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T08:12:54.185 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T08:12:54.185 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T08:12:54.186 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:53 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b[72738]: ts=2024-09-11T08:12:53.967Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T08:12:54.186 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:54 smithi196 bash[72664]: 214d90a9f27e6e0c8dbf765676b5da39631661328972f08d9bac2677f29c9325 2024-09-11T08:12:54.186 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:12:54 smithi196 systemd[1]: Started Ceph node-exporter.b for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:12:54.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:54 smithi071 ceph-mon[39897]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:54.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:54 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:54.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:54 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:54.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:54 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:54.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:54 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:54.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:54 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:54.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:54 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:12:54.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:54 smithi071 ceph-mon[25834]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:54.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:54 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:54.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:54 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:54.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:54 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:54.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:54 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:54.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:54 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:54.696 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:54 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:12:54.702 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:12:54.704 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T08:12:54.852 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:54 smithi196 ceph-mon[30677]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:54.852 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:54 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:54.853 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:54 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:54.853 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:54 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:54.853 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:54 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:54.853 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:54 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:54.853 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:54 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:12:56.087 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-11T08:12:53.076612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":963,"num_read_kb":817,"num_write":609,"num_write_kb":3237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":608,"ondisk_log_size":608,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":2386576,"kb_used_data":7732,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747541872,"statfs":{"total":767926730752,"available":765482876928,"internally_reserved":0,"allocated":7917568,"data_stored":4303984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002802"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799108+0000","last_change":"2024-09-11T08:11:25.100635+0000","last_active":"2024-09-11T08:11:38.799108+0000","last_peered":"2024-09-11T08:11:38.799108+0000","last_clean":"2024-09-11T08:11:38.799108+0000","last_became_active":"2024-09-11T08:11:25.100193+0000","last_became_peered":"2024-09-11T08:11:25.100193+0000","last_unstale":"2024-09-11T08:11:38.799108+0000","last_undegraded":"2024-09-11T08:11:38.799108+0000","last_fullsized":"2024-09-11T08:11:38.799108+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:34.434988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"67'9","reported_seq":32,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349477+0000","last_change":"2024-09-11T08:11:27.100029+0000","last_active":"2024-09-11T08:11:39.349477+0000","last_peered":"2024-09-11T08:11:39.349477+0000","last_clean":"2024-09-11T08:11:39.349477+0000","last_became_active":"2024-09-11T08:11:27.099558+0000","last_became_peered":"2024-09-11T08:11:27.099558+0000","last_unstale":"2024-09-11T08:11:39.349477+0000","last_undegraded":"2024-09-11T08:11:39.349477+0000","last_fullsized":"2024-09-11T08:11:39.349477+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:11:10.230931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.847725+0000","last_change":"2024-09-11T08:11:29.095963+0000","last_active":"2024-09-11T08:11:38.847725+0000","last_peered":"2024-09-11T08:11:38.847725+0000","last_clean":"2024-09-11T08:11:38.847725+0000","last_became_active":"2024-09-11T08:11:29.095380+0000","last_became_peered":"2024-09-11T08:11:29.095380+0000","last_unstale":"2024-09-11T08:11:38.847725+0000","last_undegraded":"2024-09-11T08:11:38.847725+0000","last_fullsized":"2024-09-11T08:11:38.847725+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:08:47.386100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348502+0000","last_change":"2024-09-11T08:11:31.103488+0000","last_active":"2024-09-11T08:11:39.348502+0000","last_peered":"2024-09-11T08:11:39.348502+0000","last_clean":"2024-09-11T08:11:39.348502+0000","last_became_active":"2024-09-11T08:11:31.103199+0000","last_became_peered":"2024-09-11T08:11:31.103199+0000","last_unstale":"2024-09-11T08:11:39.348502+0000","last_undegraded":"2024-09-11T08:11:39.348502+0000","last_fullsized":"2024-09-11T08:11:39.348502+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:31.209880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799208+0000","last_change":"2024-09-11T08:11:31.418360+0000","last_active":"2024-09-11T08:11:38.799208+0000","last_peered":"2024-09-11T08:11:38.799208+0000","last_clean":"2024-09-11T08:11:38.799208+0000","last_became_active":"2024-09-11T08:11:31.418008+0000","last_became_peered":"2024-09-11T08:11:31.418008+0000","last_unstale":"2024-09-11T08:11:38.799208+0000","last_undegraded":"2024-09-11T08:11:38.799208+0000","last_fullsized":"2024-09-11T08:11:38.799208+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:27:33.474111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.800180+0000","last_change":"2024-09-11T08:11:25.094465+0000","last_active":"2024-09-11T08:11:38.800180+0000","last_peered":"2024-09-11T08:11:38.800180+0000","last_clean":"2024-09-11T08:11:38.800180+0000","last_became_active":"2024-09-11T08:11:25.093300+0000","last_became_peered":"2024-09-11T08:11:25.093300+0000","last_unstale":"2024-09-11T08:11:38.800180+0000","last_undegraded":"2024-09-11T08:11:38.800180+0000","last_fullsized":"2024-09-11T08:11:38.800180+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:30:51.694455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"67'15","reported_seq":41,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798770+0000","last_change":"2024-09-11T08:11:27.092357+0000","last_active":"2024-09-11T08:11:38.798770+0000","last_peered":"2024-09-11T08:11:38.798770+0000","last_clean":"2024-09-11T08:11:38.798770+0000","last_became_active":"2024-09-11T08:11:27.091254+0000","last_became_peered":"2024-09-11T08:11:27.091254+0000","last_unstale":"2024-09-11T08:11:38.798770+0000","last_undegraded":"2024-09-11T08:11:38.798770+0000","last_fullsized":"2024-09-11T08:11:38.798770+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:46:04.817849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349541+0000","last_change":"2024-09-11T08:11:29.097340+0000","last_active":"2024-09-11T08:11:39.349541+0000","last_peered":"2024-09-11T08:11:39.349541+0000","last_clean":"2024-09-11T08:11:39.349541+0000","last_became_active":"2024-09-11T08:11:29.097020+0000","last_became_peered":"2024-09-11T08:11:29.097020+0000","last_unstale":"2024-09-11T08:11:39.349541+0000","last_undegraded":"2024-09-11T08:11:39.349541+0000","last_fullsized":"2024-09-11T08:11:39.349541+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:07:20.456890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798555+0000","last_change":"2024-09-11T08:11:25.090587+0000","last_active":"2024-09-11T08:11:38.798555+0000","last_peered":"2024-09-11T08:11:38.798555+0000","last_clean":"2024-09-11T08:11:38.798555+0000","last_became_active":"2024-09-11T08:11:25.090318+0000","last_became_peered":"2024-09-11T08:11:25.090318+0000","last_unstale":"2024-09-11T08:11:38.798555+0000","last_undegraded":"2024-09-11T08:11:38.798555+0000","last_fullsized":"2024-09-11T08:11:38.798555+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:44:53.045065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"67'9","reported_seq":32,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348835+0000","last_change":"2024-09-11T08:11:27.093690+0000","last_active":"2024-09-11T08:11:39.348835+0000","last_peered":"2024-09-11T08:11:39.348835+0000","last_clean":"2024-09-11T08:11:39.348835+0000","last_became_active":"2024-09-11T08:11:27.093447+0000","last_became_peered":"2024-09-11T08:11:27.093447+0000","last_unstale":"2024-09-11T08:11:39.348835+0000","last_undegraded":"2024-09-11T08:11:39.348835+0000","last_fullsized":"2024-09-11T08:11:39.348835+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:42:33.447941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798106+0000","last_change":"2024-09-11T08:11:29.100712+0000","last_active":"2024-09-11T08:11:38.798106+0000","last_peered":"2024-09-11T08:11:38.798106+0000","last_clean":"2024-09-11T08:11:38.798106+0000","last_became_active":"2024-09-11T08:11:29.100434+0000","last_became_peered":"2024-09-11T08:11:29.100434+0000","last_unstale":"2024-09-11T08:11:38.798106+0000","last_undegraded":"2024-09-11T08:11:38.798106+0000","last_fullsized":"2024-09-11T08:11:38.798106+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:15:38.503280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799014+0000","last_change":"2024-09-11T08:11:31.111239+0000","last_active":"2024-09-11T08:11:38.799014+0000","last_peered":"2024-09-11T08:11:38.799014+0000","last_clean":"2024-09-11T08:11:38.799014+0000","last_became_active":"2024-09-11T08:11:31.110886+0000","last_became_peered":"2024-09-11T08:11:31.110886+0000","last_unstale":"2024-09-11T08:11:38.799014+0000","last_undegraded":"2024-09-11T08:11:38.799014+0000","last_fullsized":"2024-09-11T08:11:38.799014+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:19:12.181717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798182+0000","last_change":"2024-09-11T08:11:25.097460+0000","last_active":"2024-09-11T08:11:38.798182+0000","last_peered":"2024-09-11T08:11:38.798182+0000","last_clean":"2024-09-11T08:11:38.798182+0000","last_became_active":"2024-09-11T08:11:25.096797+0000","last_became_peered":"2024-09-11T08:11:25.096797+0000","last_unstale":"2024-09-11T08:11:38.798182+0000","last_undegraded":"2024-09-11T08:11:38.798182+0000","last_fullsized":"2024-09-11T08:11:38.798182+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:07:00.506231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"67'5","reported_seq":26,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348930+0000","last_change":"2024-09-11T08:11:27.099207+0000","last_active":"2024-09-11T08:11:39.348930+0000","last_peered":"2024-09-11T08:11:39.348930+0000","last_clean":"2024-09-11T08:11:39.348930+0000","last_became_active":"2024-09-11T08:11:27.098976+0000","last_became_peered":"2024-09-11T08:11:27.098976+0000","last_unstale":"2024-09-11T08:11:39.348930+0000","last_undegraded":"2024-09-11T08:11:39.348930+0000","last_fullsized":"2024-09-11T08:11:39.348930+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:52.194448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348696+0000","last_change":"2024-09-11T08:11:29.095767+0000","last_active":"2024-09-11T08:11:39.348696+0000","last_peered":"2024-09-11T08:11:39.348696+0000","last_clean":"2024-09-11T08:11:39.348696+0000","last_became_active":"2024-09-11T08:11:29.095486+0000","last_became_peered":"2024-09-11T08:11:29.095486+0000","last_unstale":"2024-09-11T08:11:39.348696+0000","last_undegraded":"2024-09-11T08:11:39.348696+0000","last_fullsized":"2024-09-11T08:11:39.348696+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:19:18.172557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.797987+0000","last_change":"2024-09-11T08:11:31.106927+0000","last_active":"2024-09-11T08:11:38.797987+0000","last_peered":"2024-09-11T08:11:38.797987+0000","last_clean":"2024-09-11T08:11:38.797987+0000","last_became_active":"2024-09-11T08:11:31.106638+0000","last_became_peered":"2024-09-11T08:11:31.106638+0000","last_unstale":"2024-09-11T08:11:38.797987+0000","last_undegraded":"2024-09-11T08:11:38.797987+0000","last_fullsized":"2024-09-11T08:11:38.797987+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:05:19.586061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349464+0000","last_change":"2024-09-11T08:11:31.416861+0000","last_active":"2024-09-11T08:11:39.349464+0000","last_peered":"2024-09-11T08:11:39.349464+0000","last_clean":"2024-09-11T08:11:39.349464+0000","last_became_active":"2024-09-11T08:11:31.416214+0000","last_became_peered":"2024-09-11T08:11:31.416214+0000","last_unstale":"2024-09-11T08:11:39.349464+0000","last_undegraded":"2024-09-11T08:11:39.349464+0000","last_fullsized":"2024-09-11T08:11:39.349464+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:31:05.089040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799845+0000","last_change":"2024-09-11T08:11:25.100616+0000","last_active":"2024-09-11T08:11:38.799845+0000","last_peered":"2024-09-11T08:11:38.799845+0000","last_clean":"2024-09-11T08:11:38.799845+0000","last_became_active":"2024-09-11T08:11:25.099718+0000","last_became_peered":"2024-09-11T08:11:25.099718+0000","last_unstale":"2024-09-11T08:11:38.799845+0000","last_undegraded":"2024-09-11T08:11:38.799845+0000","last_fullsized":"2024-09-11T08:11:38.799845+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:01:55.547509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":41,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349331+0000","last_change":"2024-09-11T08:11:27.085676+0000","last_active":"2024-09-11T08:11:39.349331+0000","last_peered":"2024-09-11T08:11:39.349331+0000","last_clean":"2024-09-11T08:11:39.349331+0000","last_became_active":"2024-09-11T08:11:27.085398+0000","last_became_peered":"2024-09-11T08:11:27.085398+0000","last_unstale":"2024-09-11T08:11:39.349331+0000","last_undegraded":"2024-09-11T08:11:39.349331+0000","last_fullsized":"2024-09-11T08:11:39.349331+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:21.142564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.848283+0000","last_change":"2024-09-11T08:11:29.100692+0000","last_active":"2024-09-11T08:11:38.848283+0000","last_peered":"2024-09-11T08:11:38.848283+0000","last_clean":"2024-09-11T08:11:38.848283+0000","last_became_active":"2024-09-11T08:11:29.100287+0000","last_became_peered":"2024-09-11T08:11:29.100287+0000","last_unstale":"2024-09-11T08:11:38.848283+0000","last_undegraded":"2024-09-11T08:11:38.848283+0000","last_fullsized":"2024-09-11T08:11:38.848283+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:08:00.996144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.800362+0000","last_change":"2024-09-11T08:11:31.418085+0000","last_active":"2024-09-11T08:11:38.800362+0000","last_peered":"2024-09-11T08:11:38.800362+0000","last_clean":"2024-09-11T08:11:38.800362+0000","last_became_active":"2024-09-11T08:11:31.416348+0000","last_became_peered":"2024-09-11T08:11:31.416348+0000","last_unstale":"2024-09-11T08:11:38.800362+0000","last_undegraded":"2024-09-11T08:11:38.800362+0000","last_fullsized":"2024-09-11T08:11:38.800362+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:27:22.014521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349254+0000","last_change":"2024-09-11T08:11:25.091739+0000","last_active":"2024-09-11T08:11:39.349254+0000","last_peered":"2024-09-11T08:11:39.349254+0000","last_clean":"2024-09-11T08:11:39.349254+0000","last_became_active":"2024-09-11T08:11:25.091475+0000","last_became_peered":"2024-09-11T08:11:25.091475+0000","last_unstale":"2024-09-11T08:11:39.349254+0000","last_undegraded":"2024-09-11T08:11:39.349254+0000","last_fullsized":"2024-09-11T08:11:39.349254+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:05:39.749321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"67'12","reported_seq":39,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799797+0000","last_change":"2024-09-11T08:11:27.094286+0000","last_active":"2024-09-11T08:11:38.799797+0000","last_peered":"2024-09-11T08:11:38.799797+0000","last_clean":"2024-09-11T08:11:38.799797+0000","last_became_active":"2024-09-11T08:11:27.093647+0000","last_became_peered":"2024-09-11T08:11:27.093647+0000","last_unstale":"2024-09-11T08:11:38.799797+0000","last_undegraded":"2024-09-11T08:11:38.799797+0000","last_fullsized":"2024-09-11T08:11:38.799797+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:24:11.420124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348923+0000","last_change":"2024-09-11T08:11:29.092335+0000","last_active":"2024-09-11T08:11:39.348923+0000","last_peered":"2024-09-11T08:11:39.348923+0000","last_clean":"2024-09-11T08:11:39.348923+0000","last_became_active":"2024-09-11T08:11:29.092041+0000","last_became_peered":"2024-09-11T08:11:29.092041+0000","last_unstale":"2024-09-11T08:11:39.348923+0000","last_undegraded":"2024-09-11T08:11:39.348923+0000","last_fullsized":"2024-09-11T08:11:39.348923+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:44:47.268537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"67'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348418+0000","last_change":"2024-09-11T08:11:31.109287+0000","last_active":"2024-09-11T08:11:39.348418+0000","last_peered":"2024-09-11T08:11:39.348418+0000","last_clean":"2024-09-11T08:11:39.348418+0000","last_became_active":"2024-09-11T08:11:31.108998+0000","last_became_peered":"2024-09-11T08:11:31.108998+0000","last_unstale":"2024-09-11T08:11:39.348418+0000","last_undegraded":"2024-09-11T08:11:39.348418+0000","last_fullsized":"2024-09-11T08:11:39.348418+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:15.748936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.847982+0000","last_change":"2024-09-11T08:11:25.085876+0000","last_active":"2024-09-11T08:11:38.847982+0000","last_peered":"2024-09-11T08:11:38.847982+0000","last_clean":"2024-09-11T08:11:38.847982+0000","last_became_active":"2024-09-11T08:11:25.085605+0000","last_became_peered":"2024-09-11T08:11:25.085605+0000","last_unstale":"2024-09-11T08:11:38.847982+0000","last_undegraded":"2024-09-11T08:11:38.847982+0000","last_fullsized":"2024-09-11T08:11:38.847982+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:58:58.546112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"67'10","reported_seq":31,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799589+0000","last_change":"2024-09-11T08:11:27.094925+0000","last_active":"2024-09-11T08:11:38.799589+0000","last_peered":"2024-09-11T08:11:38.799589+0000","last_clean":"2024-09-11T08:11:38.799589+0000","last_became_active":"2024-09-11T08:11:27.094637+0000","last_became_peered":"2024-09-11T08:11:27.094637+0000","last_unstale":"2024-09-11T08:11:38.799589+0000","last_undegraded":"2024-09-11T08:11:38.799589+0000","last_fullsized":"2024-09-11T08:11:38.799589+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:47.132046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"67'8","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:43.408598+0000","last_change":"2024-09-11T08:11:29.100326+0000","last_active":"2024-09-11T08:12:43.408598+0000","last_peered":"2024-09-11T08:12:43.408598+0000","last_clean":"2024-09-11T08:12:43.408598+0000","last_became_active":"2024-09-11T08:11:29.100000+0000","last_became_peered":"2024-09-11T08:11:29.100000+0000","last_unstale":"2024-09-11T08:12:43.408598+0000","last_undegraded":"2024-09-11T08:12:43.408598+0000","last_fullsized":"2024-09-11T08:12:43.408598+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:46.095412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":41,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.848552+0000","last_change":"2024-09-11T08:11:27.099619+0000","last_active":"2024-09-11T08:11:38.848552+0000","last_peered":"2024-09-11T08:11:38.848552+0000","last_clean":"2024-09-11T08:11:38.848552+0000","last_became_active":"2024-09-11T08:11:27.099102+0000","last_became_peered":"2024-09-11T08:11:27.099102+0000","last_unstale":"2024-09-11T08:11:38.848552+0000","last_undegraded":"2024-09-11T08:11:38.848552+0000","last_fullsized":"2024-09-11T08:11:38.848552+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:48.511606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799396+0000","last_change":"2024-09-11T08:11:25.097018+0000","last_active":"2024-09-11T08:11:38.799396+0000","last_peered":"2024-09-11T08:11:38.799396+0000","last_clean":"2024-09-11T08:11:38.799396+0000","last_became_active":"2024-09-11T08:11:25.096771+0000","last_became_peered":"2024-09-11T08:11:25.096771+0000","last_unstale":"2024-09-11T08:11:38.799396+0000","last_undegraded":"2024-09-11T08:11:38.799396+0000","last_fullsized":"2024-09-11T08:11:38.799396+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:17:49.864799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":39,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:43.408588+0000","last_change":"2024-09-11T08:11:29.102472+0000","last_active":"2024-09-11T08:12:43.408588+0000","last_peered":"2024-09-11T08:12:43.408588+0000","last_clean":"2024-09-11T08:12:43.408588+0000","last_became_active":"2024-09-11T08:11:29.101756+0000","last_became_peered":"2024-09-11T08:11:29.101756+0000","last_unstale":"2024-09-11T08:12:43.408588+0000","last_undegraded":"2024-09-11T08:12:43.408588+0000","last_fullsized":"2024-09-11T08:12:43.408588+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:58:44.331961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798190+0000","last_change":"2024-09-11T08:11:31.108596+0000","last_active":"2024-09-11T08:11:38.798190+0000","last_peered":"2024-09-11T08:11:38.798190+0000","last_clean":"2024-09-11T08:11:38.798190+0000","last_became_active":"2024-09-11T08:11:31.108272+0000","last_became_peered":"2024-09-11T08:11:31.108272+0000","last_unstale":"2024-09-11T08:11:38.798190+0000","last_undegraded":"2024-09-11T08:11:38.798190+0000","last_fullsized":"2024-09-11T08:11:38.798190+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:01:04.360030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"67'18","reported_seq":48,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799182+0000","last_change":"2024-09-11T08:11:27.093207+0000","last_active":"2024-09-11T08:11:38.799182+0000","last_peered":"2024-09-11T08:11:38.799182+0000","last_clean":"2024-09-11T08:11:38.799182+0000","last_became_active":"2024-09-11T08:11:27.091391+0000","last_became_peered":"2024-09-11T08:11:27.091391+0000","last_unstale":"2024-09-11T08:11:38.799182+0000","last_undegraded":"2024-09-11T08:11:38.799182+0000","last_fullsized":"2024-09-11T08:11:38.799182+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:31:52.841914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.800478+0000","last_change":"2024-09-11T08:11:25.095035+0000","last_active":"2024-09-11T08:11:38.800478+0000","last_peered":"2024-09-11T08:11:38.800478+0000","last_clean":"2024-09-11T08:11:38.800478+0000","last_became_active":"2024-09-11T08:11:25.094075+0000","last_became_peered":"2024-09-11T08:11:25.094075+0000","last_unstale":"2024-09-11T08:11:38.800478+0000","last_undegraded":"2024-09-11T08:11:38.800478+0000","last_fullsized":"2024-09-11T08:11:38.800478+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:23.648793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349800+0000","last_change":"2024-09-11T08:11:29.091792+0000","last_active":"2024-09-11T08:11:39.349800+0000","last_peered":"2024-09-11T08:11:39.349800+0000","last_clean":"2024-09-11T08:11:39.349800+0000","last_became_active":"2024-09-11T08:11:29.091483+0000","last_became_peered":"2024-09-11T08:11:29.091483+0000","last_unstale":"2024-09-11T08:11:39.349800+0000","last_undegraded":"2024-09-11T08:11:39.349800+0000","last_fullsized":"2024-09-11T08:11:39.349800+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:17.519130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348399+0000","last_change":"2024-09-11T08:11:31.106932+0000","last_active":"2024-09-11T08:11:39.348399+0000","last_peered":"2024-09-11T08:11:39.348399+0000","last_clean":"2024-09-11T08:11:39.348399+0000","last_became_active":"2024-09-11T08:11:31.105616+0000","last_became_peered":"2024-09-11T08:11:31.105616+0000","last_unstale":"2024-09-11T08:11:39.348399+0000","last_undegraded":"2024-09-11T08:11:39.348399+0000","last_fullsized":"2024-09-11T08:11:39.348399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:44:44.660887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"67'14","reported_seq":37,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349887+0000","last_change":"2024-09-11T08:11:27.092287+0000","last_active":"2024-09-11T08:11:39.349887+0000","last_peered":"2024-09-11T08:11:39.349887+0000","last_clean":"2024-09-11T08:11:39.349887+0000","last_became_active":"2024-09-11T08:11:27.091956+0000","last_became_peered":"2024-09-11T08:11:27.091956+0000","last_unstale":"2024-09-11T08:11:39.349887+0000","last_undegraded":"2024-09-11T08:11:39.349887+0000","last_fullsized":"2024-09-11T08:11:39.349887+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:11:37.593994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"60'1","reported_seq":27,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799845+0000","last_change":"2024-09-11T08:11:25.097889+0000","last_active":"2024-09-11T08:11:38.799845+0000","last_peered":"2024-09-11T08:11:38.799845+0000","last_clean":"2024-09-11T08:11:38.799845+0000","last_became_active":"2024-09-11T08:11:25.096358+0000","last_became_peered":"2024-09-11T08:11:25.096358+0000","last_unstale":"2024-09-11T08:11:38.799845+0000","last_undegraded":"2024-09-11T08:11:38.799845+0000","last_fullsized":"2024-09-11T08:11:38.799845+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:21.009658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":39,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:43.408487+0000","last_change":"2024-09-11T08:11:29.096174+0000","last_active":"2024-09-11T08:12:43.408487+0000","last_peered":"2024-09-11T08:12:43.408487+0000","last_clean":"2024-09-11T08:12:43.408487+0000","last_became_active":"2024-09-11T08:11:29.095893+0000","last_became_peered":"2024-09-11T08:11:29.095893+0000","last_unstale":"2024-09-11T08:12:43.408487+0000","last_undegraded":"2024-09-11T08:12:43.408487+0000","last_fullsized":"2024-09-11T08:12:43.408487+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:36:14.701080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348974+0000","last_change":"2024-09-11T08:11:31.418326+0000","last_active":"2024-09-11T08:11:39.348974+0000","last_peered":"2024-09-11T08:11:39.348974+0000","last_clean":"2024-09-11T08:11:39.348974+0000","last_became_active":"2024-09-11T08:11:31.417976+0000","last_became_peered":"2024-09-11T08:11:31.417976+0000","last_unstale":"2024-09-11T08:11:39.348974+0000","last_undegraded":"2024-09-11T08:11:39.348974+0000","last_fullsized":"2024-09-11T08:11:39.348974+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:34:48.447713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"67'10","reported_seq":31,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.848106+0000","last_change":"2024-09-11T08:11:27.094719+0000","last_active":"2024-09-11T08:11:38.848106+0000","last_peered":"2024-09-11T08:11:38.848106+0000","last_clean":"2024-09-11T08:11:38.848106+0000","last_became_active":"2024-09-11T08:11:27.093081+0000","last_became_peered":"2024-09-11T08:11:27.093081+0000","last_unstale":"2024-09-11T08:11:38.848106+0000","last_undegraded":"2024-09-11T08:11:38.848106+0000","last_fullsized":"2024-09-11T08:11:38.848106+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:28:27.045201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798617+0000","last_change":"2024-09-11T08:11:25.089227+0000","last_active":"2024-09-11T08:11:38.798617+0000","last_peered":"2024-09-11T08:11:38.798617+0000","last_clean":"2024-09-11T08:11:38.798617+0000","last_became_active":"2024-09-11T08:11:25.088974+0000","last_became_peered":"2024-09-11T08:11:25.088974+0000","last_unstale":"2024-09-11T08:11:38.798617+0000","last_undegraded":"2024-09-11T08:11:38.798617+0000","last_fullsized":"2024-09-11T08:11:38.798617+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:12:49.783167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"67'8","reported_seq":39,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:43.408488+0000","last_change":"2024-09-11T08:11:29.099049+0000","last_active":"2024-09-11T08:12:43.408488+0000","last_peered":"2024-09-11T08:12:43.408488+0000","last_clean":"2024-09-11T08:12:43.408488+0000","last_became_active":"2024-09-11T08:11:29.098760+0000","last_became_peered":"2024-09-11T08:11:29.098760+0000","last_unstale":"2024-09-11T08:12:43.408488+0000","last_undegraded":"2024-09-11T08:12:43.408488+0000","last_fullsized":"2024-09-11T08:12:43.408488+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:11:46.784852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798598+0000","last_change":"2024-09-11T08:11:31.114406+0000","last_active":"2024-09-11T08:11:38.798598+0000","last_peered":"2024-09-11T08:11:38.798598+0000","last_clean":"2024-09-11T08:11:38.798598+0000","last_became_active":"2024-09-11T08:11:31.114102+0000","last_became_peered":"2024-09-11T08:11:31.114102+0000","last_unstale":"2024-09-11T08:11:38.798598+0000","last_undegraded":"2024-09-11T08:11:38.798598+0000","last_fullsized":"2024-09-11T08:11:38.798598+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:46:04.586606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"67'19","reported_seq":52,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799762+0000","last_change":"2024-09-11T08:11:27.091494+0000","last_active":"2024-09-11T08:11:38.799762+0000","last_peered":"2024-09-11T08:11:38.799762+0000","last_clean":"2024-09-11T08:11:38.799762+0000","last_became_active":"2024-09-11T08:11:27.091145+0000","last_became_peered":"2024-09-11T08:11:27.091145+0000","last_unstale":"2024-09-11T08:11:38.799762+0000","last_undegraded":"2024-09-11T08:11:38.799762+0000","last_fullsized":"2024-09-11T08:11:38.799762+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:29:37.648590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.848451+0000","last_change":"2024-09-11T08:11:25.096402+0000","last_active":"2024-09-11T08:11:38.848451+0000","last_peered":"2024-09-11T08:11:38.848451+0000","last_clean":"2024-09-11T08:11:38.848451+0000","last_became_active":"2024-09-11T08:11:25.096135+0000","last_became_peered":"2024-09-11T08:11:25.096135+0000","last_unstale":"2024-09-11T08:11:38.848451+0000","last_undegraded":"2024-09-11T08:11:38.848451+0000","last_fullsized":"2024-09-11T08:11:38.848451+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:18.478984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.808199+0000","last_change":"2024-09-11T08:11:29.099261+0000","last_active":"2024-09-11T08:11:38.808199+0000","last_peered":"2024-09-11T08:11:38.808199+0000","last_clean":"2024-09-11T08:11:38.808199+0000","last_became_active":"2024-09-11T08:11:29.098974+0000","last_became_peered":"2024-09-11T08:11:29.098974+0000","last_unstale":"2024-09-11T08:11:38.808199+0000","last_undegraded":"2024-09-11T08:11:38.808199+0000","last_fullsized":"2024-09-11T08:11:38.808199+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:47:51.029612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.847668+0000","last_change":"2024-09-11T08:11:31.416917+0000","last_active":"2024-09-11T08:11:38.847668+0000","last_peered":"2024-09-11T08:11:38.847668+0000","last_clean":"2024-09-11T08:11:38.847668+0000","last_became_active":"2024-09-11T08:11:31.416538+0000","last_became_peered":"2024-09-11T08:11:31.416538+0000","last_unstale":"2024-09-11T08:11:38.847668+0000","last_undegraded":"2024-09-11T08:11:38.847668+0000","last_fullsized":"2024-09-11T08:11:38.847668+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:38:14.247963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"67'28","reported_seq":81,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:43.408493+0000","last_change":"2024-09-11T08:11:27.097248+0000","last_active":"2024-09-11T08:12:43.408493+0000","last_peered":"2024-09-11T08:12:43.408493+0000","last_clean":"2024-09-11T08:12:43.408493+0000","last_became_active":"2024-09-11T08:11:27.096947+0000","last_became_peered":"2024-09-11T08:11:27.096947+0000","last_unstale":"2024-09-11T08:12:43.408493+0000","last_undegraded":"2024-09-11T08:12:43.408493+0000","last_fullsized":"2024-09-11T08:12:43.408493+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:51.723234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349228+0000","last_change":"2024-09-11T08:11:25.093960+0000","last_active":"2024-09-11T08:11:39.349228+0000","last_peered":"2024-09-11T08:11:39.349228+0000","last_clean":"2024-09-11T08:11:39.349228+0000","last_became_active":"2024-09-11T08:11:25.093461+0000","last_became_peered":"2024-09-11T08:11:25.093461+0000","last_unstale":"2024-09-11T08:11:39.349228+0000","last_undegraded":"2024-09-11T08:11:39.349228+0000","last_fullsized":"2024-09-11T08:11:39.349228+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:36.271963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"64'2","reported_seq":25,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798430+0000","last_change":"2024-09-11T08:11:29.091587+0000","last_active":"2024-09-11T08:11:38.798430+0000","last_peered":"2024-09-11T08:11:38.798430+0000","last_clean":"2024-09-11T08:11:38.798430+0000","last_became_active":"2024-09-11T08:11:25.096910+0000","last_became_peered":"2024-09-11T08:11:25.096910+0000","last_unstale":"2024-09-11T08:11:38.798430+0000","last_undegraded":"2024-09-11T08:11:38.798430+0000","last_fullsized":"2024-09-11T08:11:38.798430+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:03.774605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001668763,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798973+0000","last_change":"2024-09-11T08:11:29.099919+0000","last_active":"2024-09-11T08:11:38.798973+0000","last_peered":"2024-09-11T08:11:38.798973+0000","last_clean":"2024-09-11T08:11:38.798973+0000","last_became_active":"2024-09-11T08:11:29.099262+0000","last_became_peered":"2024-09-11T08:11:29.099262+0000","last_unstale":"2024-09-11T08:11:38.798973+0000","last_undegraded":"2024-09-11T08:11:38.798973+0000","last_fullsized":"2024-09-11T08:11:38.798973+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:21:38.486568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.800074+0000","last_change":"2024-09-11T08:11:31.105837+0000","last_active":"2024-09-11T08:11:38.800074+0000","last_peered":"2024-09-11T08:11:38.800074+0000","last_clean":"2024-09-11T08:11:38.800074+0000","last_became_active":"2024-09-11T08:11:31.105565+0000","last_became_peered":"2024-09-11T08:11:31.105565+0000","last_unstale":"2024-09-11T08:11:38.800074+0000","last_undegraded":"2024-09-11T08:11:38.800074+0000","last_fullsized":"2024-09-11T08:11:38.800074+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:09:17.153498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"67'13","reported_seq":43,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.848540+0000","last_change":"2024-09-11T08:11:27.095117+0000","last_active":"2024-09-11T08:11:38.848540+0000","last_peered":"2024-09-11T08:11:38.848540+0000","last_clean":"2024-09-11T08:11:38.848540+0000","last_became_active":"2024-09-11T08:11:27.094824+0000","last_became_peered":"2024-09-11T08:11:27.094824+0000","last_unstale":"2024-09-11T08:11:38.848540+0000","last_undegraded":"2024-09-11T08:11:38.848540+0000","last_fullsized":"2024-09-11T08:11:38.848540+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:15.592489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.848040+0000","last_change":"2024-09-11T08:11:25.092824+0000","last_active":"2024-09-11T08:11:38.848040+0000","last_peered":"2024-09-11T08:11:38.848040+0000","last_clean":"2024-09-11T08:11:38.848040+0000","last_became_active":"2024-09-11T08:11:25.092401+0000","last_became_peered":"2024-09-11T08:11:25.092401+0000","last_unstale":"2024-09-11T08:11:38.848040+0000","last_undegraded":"2024-09-11T08:11:38.848040+0000","last_fullsized":"2024-09-11T08:11:38.848040+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:54.500772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"62'1","reported_seq":24,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348726+0000","last_change":"2024-09-11T08:11:29.082405+0000","last_active":"2024-09-11T08:11:39.348726+0000","last_peered":"2024-09-11T08:11:39.348726+0000","last_clean":"2024-09-11T08:11:39.348726+0000","last_became_active":"2024-09-11T08:11:25.094366+0000","last_became_peered":"2024-09-11T08:11:25.094366+0000","last_unstale":"2024-09-11T08:11:39.348726+0000","last_undegraded":"2024-09-11T08:11:39.348726+0000","last_fullsized":"2024-09-11T08:11:39.348726+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:41:06.586281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000367742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349220+0000","last_change":"2024-09-11T08:11:29.096678+0000","last_active":"2024-09-11T08:11:39.349220+0000","last_peered":"2024-09-11T08:11:39.349220+0000","last_clean":"2024-09-11T08:11:39.349220+0000","last_became_active":"2024-09-11T08:11:29.096397+0000","last_became_peered":"2024-09-11T08:11:29.096397+0000","last_unstale":"2024-09-11T08:11:39.349220+0000","last_undegraded":"2024-09-11T08:11:39.349220+0000","last_fullsized":"2024-09-11T08:11:39.349220+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:26.149855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349068+0000","last_change":"2024-09-11T08:11:31.416782+0000","last_active":"2024-09-11T08:11:39.349068+0000","last_peered":"2024-09-11T08:11:39.349068+0000","last_clean":"2024-09-11T08:11:39.349068+0000","last_became_active":"2024-09-11T08:11:31.416481+0000","last_became_peered":"2024-09-11T08:11:31.416481+0000","last_unstale":"2024-09-11T08:11:39.349068+0000","last_undegraded":"2024-09-11T08:11:39.349068+0000","last_fullsized":"2024-09-11T08:11:39.349068+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:39:09.994375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"67'12","reported_seq":34,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799672+0000","last_change":"2024-09-11T08:11:27.096351+0000","last_active":"2024-09-11T08:11:38.799672+0000","last_peered":"2024-09-11T08:11:38.799672+0000","last_clean":"2024-09-11T08:11:38.799672+0000","last_became_active":"2024-09-11T08:11:27.096081+0000","last_became_peered":"2024-09-11T08:11:27.096081+0000","last_unstale":"2024-09-11T08:11:38.799672+0000","last_undegraded":"2024-09-11T08:11:38.799672+0000","last_fullsized":"2024-09-11T08:11:38.799672+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:55:58.950174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798710+0000","last_change":"2024-09-11T08:11:25.098694+0000","last_active":"2024-09-11T08:11:38.798710+0000","last_peered":"2024-09-11T08:11:38.798710+0000","last_clean":"2024-09-11T08:11:38.798710+0000","last_became_active":"2024-09-11T08:11:25.098070+0000","last_became_peered":"2024-09-11T08:11:25.098070+0000","last_unstale":"2024-09-11T08:11:38.798710+0000","last_undegraded":"2024-09-11T08:11:38.798710+0000","last_fullsized":"2024-09-11T08:11:38.798710+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:00:49.357459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":106,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:50.780252+0000","last_change":"2024-09-11T08:11:29.089234+0000","last_active":"2024-09-11T08:12:50.780252+0000","last_peered":"2024-09-11T08:12:50.780252+0000","last_clean":"2024-09-11T08:12:50.780252+0000","last_became_active":"2024-09-11T08:11:25.094995+0000","last_became_peered":"2024-09-11T08:11:25.094995+0000","last_unstale":"2024-09-11T08:12:50.780252+0000","last_undegraded":"2024-09-11T08:12:50.780252+0000","last_fullsized":"2024-09-11T08:12:50.780252+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:43:19.613394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000316562,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":80,"num_read_kb":75,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798610+0000","last_change":"2024-09-11T08:11:29.099690+0000","last_active":"2024-09-11T08:11:38.798610+0000","last_peered":"2024-09-11T08:11:38.798610+0000","last_clean":"2024-09-11T08:11:38.798610+0000","last_became_active":"2024-09-11T08:11:29.099315+0000","last_became_peered":"2024-09-11T08:11:29.099315+0000","last_unstale":"2024-09-11T08:11:38.798610+0000","last_undegraded":"2024-09-11T08:11:38.798610+0000","last_fullsized":"2024-09-11T08:11:38.798610+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:52.867239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.847853+0000","last_change":"2024-09-11T08:11:31.106743+0000","last_active":"2024-09-11T08:11:38.847853+0000","last_peered":"2024-09-11T08:11:38.847853+0000","last_clean":"2024-09-11T08:11:38.847853+0000","last_became_active":"2024-09-11T08:11:31.106471+0000","last_became_peered":"2024-09-11T08:11:31.106471+0000","last_unstale":"2024-09-11T08:11:38.847853+0000","last_undegraded":"2024-09-11T08:11:38.847853+0000","last_fullsized":"2024-09-11T08:11:38.847853+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:52:16.190282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":38,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.808621+0000","last_change":"2024-09-11T08:11:27.089055+0000","last_active":"2024-09-11T08:11:38.808621+0000","last_peered":"2024-09-11T08:11:38.808621+0000","last_clean":"2024-09-11T08:11:38.808621+0000","last_became_active":"2024-09-11T08:11:27.088754+0000","last_became_peered":"2024-09-11T08:11:27.088754+0000","last_unstale":"2024-09-11T08:11:38.808621+0000","last_undegraded":"2024-09-11T08:11:38.808621+0000","last_fullsized":"2024-09-11T08:11:38.808621+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:49:34.528176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799094+0000","last_change":"2024-09-11T08:11:25.091612+0000","last_active":"2024-09-11T08:11:38.799094+0000","last_peered":"2024-09-11T08:11:38.799094+0000","last_clean":"2024-09-11T08:11:38.799094+0000","last_became_active":"2024-09-11T08:11:25.091216+0000","last_became_peered":"2024-09-11T08:11:25.091216+0000","last_unstale":"2024-09-11T08:11:38.799094+0000","last_undegraded":"2024-09-11T08:11:38.799094+0000","last_fullsized":"2024-09-11T08:11:38.799094+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:43:14.681940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"69'139","reported_seq":271,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:03.115868+0000","last_change":"2024-09-11T08:11:16.050351+0000","last_active":"2024-09-11T08:12:03.115868+0000","last_peered":"2024-09-11T08:12:03.115868+0000","last_clean":"2024-09-11T08:12:03.115868+0000","last_became_active":"2024-09-11T08:11:16.049986+0000","last_became_peered":"2024-09-11T08:11:16.049986+0000","last_unstale":"2024-09-11T08:12:03.115868+0000","last_undegraded":"2024-09-11T08:12:03.115868+0000","last_fullsized":"2024-09-11T08:12:03.115868+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:09:13.980611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:09:13.980611+0000","last_clean_scrub_stamp":"2024-09-11T08:09:13.980611+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:36:46.350373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":258,"num_write":179,"num_write_kb":3194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349570+0000","last_change":"2024-09-11T08:11:29.095149+0000","last_active":"2024-09-11T08:11:39.349570+0000","last_peered":"2024-09-11T08:11:39.349570+0000","last_clean":"2024-09-11T08:11:39.349570+0000","last_became_active":"2024-09-11T08:11:29.094835+0000","last_became_peered":"2024-09-11T08:11:29.094835+0000","last_unstale":"2024-09-11T08:11:39.349570+0000","last_undegraded":"2024-09-11T08:11:39.349570+0000","last_fullsized":"2024-09-11T08:11:39.349570+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:03.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798647+0000","last_change":"2024-09-11T08:11:31.105546+0000","last_active":"2024-09-11T08:11:38.798647+0000","last_peered":"2024-09-11T08:11:38.798647+0000","last_clean":"2024-09-11T08:11:38.798647+0000","last_became_active":"2024-09-11T08:11:31.105285+0000","last_became_peered":"2024-09-11T08:11:31.105285+0000","last_unstale":"2024-09-11T08:11:38.798647+0000","last_undegraded":"2024-09-11T08:11:38.798647+0000","last_fullsized":"2024-09-11T08:11:38.798647+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:03:01.327591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"67'11","reported_seq":35,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349620+0000","last_change":"2024-09-11T08:11:27.099851+0000","last_active":"2024-09-11T08:11:39.349620+0000","last_peered":"2024-09-11T08:11:39.349620+0000","last_clean":"2024-09-11T08:11:39.349620+0000","last_became_active":"2024-09-11T08:11:27.098448+0000","last_became_peered":"2024-09-11T08:11:27.098448+0000","last_unstale":"2024-09-11T08:11:39.349620+0000","last_undegraded":"2024-09-11T08:11:39.349620+0000","last_fullsized":"2024-09-11T08:11:39.349620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:40.555678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.350018+0000","last_change":"2024-09-11T08:11:25.090100+0000","last_active":"2024-09-11T08:11:39.350018+0000","last_peered":"2024-09-11T08:11:39.350018+0000","last_clean":"2024-09-11T08:11:39.350018+0000","last_became_active":"2024-09-11T08:11:25.089810+0000","last_became_peered":"2024-09-11T08:11:25.089810+0000","last_unstale":"2024-09-11T08:11:39.350018+0000","last_undegraded":"2024-09-11T08:11:39.350018+0000","last_fullsized":"2024-09-11T08:11:39.350018+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:00:15.181451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798476+0000","last_change":"2024-09-11T08:11:29.099677+0000","last_active":"2024-09-11T08:11:38.798476+0000","last_peered":"2024-09-11T08:11:38.798476+0000","last_clean":"2024-09-11T08:11:38.798476+0000","last_became_active":"2024-09-11T08:11:29.099339+0000","last_became_peered":"2024-09-11T08:11:29.099339+0000","last_unstale":"2024-09-11T08:11:38.798476+0000","last_undegraded":"2024-09-11T08:11:38.798476+0000","last_fullsized":"2024-09-11T08:11:38.798476+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:38.895315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799625+0000","last_change":"2024-09-11T08:11:31.416874+0000","last_active":"2024-09-11T08:11:38.799625+0000","last_peered":"2024-09-11T08:11:38.799625+0000","last_clean":"2024-09-11T08:11:38.799625+0000","last_became_active":"2024-09-11T08:11:31.416491+0000","last_became_peered":"2024-09-11T08:11:31.416491+0000","last_unstale":"2024-09-11T08:11:38.799625+0000","last_undegraded":"2024-09-11T08:11:38.799625+0000","last_fullsized":"2024-09-11T08:11:38.799625+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:38:00.752222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":44,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349976+0000","last_change":"2024-09-11T08:11:27.099368+0000","last_active":"2024-09-11T08:11:39.349976+0000","last_peered":"2024-09-11T08:11:39.349976+0000","last_clean":"2024-09-11T08:11:39.349976+0000","last_became_active":"2024-09-11T08:11:27.098777+0000","last_became_peered":"2024-09-11T08:11:27.098777+0000","last_unstale":"2024-09-11T08:11:39.349976+0000","last_undegraded":"2024-09-11T08:11:39.349976+0000","last_fullsized":"2024-09-11T08:11:39.349976+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:31.994742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"60'1","reported_seq":25,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.808411+0000","last_change":"2024-09-11T08:11:25.088734+0000","last_active":"2024-09-11T08:11:38.808411+0000","last_peered":"2024-09-11T08:11:38.808411+0000","last_clean":"2024-09-11T08:11:38.808411+0000","last_became_active":"2024-09-11T08:11:25.088439+0000","last_became_peered":"2024-09-11T08:11:25.088439+0000","last_unstale":"2024-09-11T08:11:38.808411+0000","last_undegraded":"2024-09-11T08:11:38.808411+0000","last_fullsized":"2024-09-11T08:11:38.808411+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:10:43.793494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.847913+0000","last_change":"2024-09-11T08:11:29.099441+0000","last_active":"2024-09-11T08:11:38.847913+0000","last_peered":"2024-09-11T08:11:38.847913+0000","last_clean":"2024-09-11T08:11:38.847913+0000","last_became_active":"2024-09-11T08:11:29.099164+0000","last_became_peered":"2024-09-11T08:11:29.099164+0000","last_unstale":"2024-09-11T08:11:38.847913+0000","last_undegraded":"2024-09-11T08:11:38.847913+0000","last_fullsized":"2024-09-11T08:11:38.847913+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:44:03.366079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348287+0000","last_change":"2024-09-11T08:11:31.111478+0000","last_active":"2024-09-11T08:11:39.348287+0000","last_peered":"2024-09-11T08:11:39.348287+0000","last_clean":"2024-09-11T08:11:39.348287+0000","last_became_active":"2024-09-11T08:11:31.111172+0000","last_became_peered":"2024-09-11T08:11:31.111172+0000","last_unstale":"2024-09-11T08:11:39.348287+0000","last_undegraded":"2024-09-11T08:11:39.348287+0000","last_fullsized":"2024-09-11T08:11:39.348287+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:19.616502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":31,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349011+0000","last_change":"2024-09-11T08:11:27.097114+0000","last_active":"2024-09-11T08:11:39.349011+0000","last_peered":"2024-09-11T08:11:39.349011+0000","last_clean":"2024-09-11T08:11:39.349011+0000","last_became_active":"2024-09-11T08:11:27.096791+0000","last_became_peered":"2024-09-11T08:11:27.096791+0000","last_unstale":"2024-09-11T08:11:39.349011+0000","last_undegraded":"2024-09-11T08:11:39.349011+0000","last_fullsized":"2024-09-11T08:11:39.349011+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:58.714797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799591+0000","last_change":"2024-09-11T08:11:25.095622+0000","last_active":"2024-09-11T08:11:38.799591+0000","last_peered":"2024-09-11T08:11:38.799591+0000","last_clean":"2024-09-11T08:11:38.799591+0000","last_became_active":"2024-09-11T08:11:25.095374+0000","last_became_peered":"2024-09-11T08:11:25.095374+0000","last_unstale":"2024-09-11T08:11:38.799591+0000","last_undegraded":"2024-09-11T08:11:38.799591+0000","last_fullsized":"2024-09-11T08:11:38.799591+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:49.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":42,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:43.408492+0000","last_change":"2024-09-11T08:11:29.097376+0000","last_active":"2024-09-11T08:12:43.408492+0000","last_peered":"2024-09-11T08:12:43.408492+0000","last_clean":"2024-09-11T08:12:43.408492+0000","last_became_active":"2024-09-11T08:11:29.097072+0000","last_became_peered":"2024-09-11T08:11:29.097072+0000","last_unstale":"2024-09-11T08:12:43.408492+0000","last_undegraded":"2024-09-11T08:12:43.408492+0000","last_fullsized":"2024-09-11T08:12:43.408492+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:37:17.423880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349308+0000","last_change":"2024-09-11T08:11:31.112304+0000","last_active":"2024-09-11T08:11:39.349308+0000","last_peered":"2024-09-11T08:11:39.349308+0000","last_clean":"2024-09-11T08:11:39.349308+0000","last_became_active":"2024-09-11T08:11:31.112003+0000","last_became_peered":"2024-09-11T08:11:31.112003+0000","last_unstale":"2024-09-11T08:11:39.349308+0000","last_undegraded":"2024-09-11T08:11:39.349308+0000","last_fullsized":"2024-09-11T08:11:39.349308+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:20:31.627281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"67'9","reported_seq":32,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799680+0000","last_change":"2024-09-11T08:11:27.090518+0000","last_active":"2024-09-11T08:11:38.799680+0000","last_peered":"2024-09-11T08:11:38.799680+0000","last_clean":"2024-09-11T08:11:38.799680+0000","last_became_active":"2024-09-11T08:11:27.090227+0000","last_became_peered":"2024-09-11T08:11:27.090227+0000","last_unstale":"2024-09-11T08:11:38.799680+0000","last_undegraded":"2024-09-11T08:11:38.799680+0000","last_fullsized":"2024-09-11T08:11:38.799680+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:13.167762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798293+0000","last_change":"2024-09-11T08:11:25.093590+0000","last_active":"2024-09-11T08:11:38.798293+0000","last_peered":"2024-09-11T08:11:38.798293+0000","last_clean":"2024-09-11T08:11:38.798293+0000","last_became_active":"2024-09-11T08:11:25.093285+0000","last_became_peered":"2024-09-11T08:11:25.093285+0000","last_unstale":"2024-09-11T08:11:38.798293+0000","last_undegraded":"2024-09-11T08:11:38.798293+0000","last_fullsized":"2024-09-11T08:11:38.798293+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:06:09.251490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348593+0000","last_change":"2024-09-11T08:11:29.092048+0000","last_active":"2024-09-11T08:11:39.348593+0000","last_peered":"2024-09-11T08:11:39.348593+0000","last_clean":"2024-09-11T08:11:39.348593+0000","last_became_active":"2024-09-11T08:11:29.091748+0000","last_became_peered":"2024-09-11T08:11:29.091748+0000","last_unstale":"2024-09-11T08:11:39.348593+0000","last_undegraded":"2024-09-11T08:11:39.348593+0000","last_fullsized":"2024-09-11T08:11:39.348593+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:59:59.797833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798653+0000","last_change":"2024-09-11T08:11:31.114435+0000","last_active":"2024-09-11T08:11:38.798653+0000","last_peered":"2024-09-11T08:11:38.798653+0000","last_clean":"2024-09-11T08:11:38.798653+0000","last_became_active":"2024-09-11T08:11:31.114063+0000","last_became_peered":"2024-09-11T08:11:31.114063+0000","last_unstale":"2024-09-11T08:11:38.798653+0000","last_undegraded":"2024-09-11T08:11:38.798653+0000","last_fullsized":"2024-09-11T08:11:38.798653+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:08:59.278736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"67'19","reported_seq":47,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.808320+0000","last_change":"2024-09-11T08:11:27.089754+0000","last_active":"2024-09-11T08:11:38.808320+0000","last_peered":"2024-09-11T08:11:38.808320+0000","last_clean":"2024-09-11T08:11:38.808320+0000","last_became_active":"2024-09-11T08:11:27.089090+0000","last_became_peered":"2024-09-11T08:11:27.089090+0000","last_unstale":"2024-09-11T08:11:38.808320+0000","last_undegraded":"2024-09-11T08:11:38.808320+0000","last_fullsized":"2024-09-11T08:11:38.808320+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:18:32.773640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349374+0000","last_change":"2024-09-11T08:11:25.090325+0000","last_active":"2024-09-11T08:11:39.349374+0000","last_peered":"2024-09-11T08:11:39.349374+0000","last_clean":"2024-09-11T08:11:39.349374+0000","last_became_active":"2024-09-11T08:11:25.090057+0000","last_became_peered":"2024-09-11T08:11:25.090057+0000","last_unstale":"2024-09-11T08:11:39.349374+0000","last_undegraded":"2024-09-11T08:11:39.349374+0000","last_fullsized":"2024-09-11T08:11:39.349374+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:57.545046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348496+0000","last_change":"2024-09-11T08:11:29.099277+0000","last_active":"2024-09-11T08:11:39.348496+0000","last_peered":"2024-09-11T08:11:39.348496+0000","last_clean":"2024-09-11T08:11:39.348496+0000","last_became_active":"2024-09-11T08:11:29.098960+0000","last_became_peered":"2024-09-11T08:11:29.098960+0000","last_unstale":"2024-09-11T08:11:39.348496+0000","last_undegraded":"2024-09-11T08:11:39.348496+0000","last_fullsized":"2024-09-11T08:11:39.348496+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:40:53.810999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348485+0000","last_change":"2024-09-11T08:11:31.105821+0000","last_active":"2024-09-11T08:11:39.348485+0000","last_peered":"2024-09-11T08:11:39.348485+0000","last_clean":"2024-09-11T08:11:39.348485+0000","last_became_active":"2024-09-11T08:11:31.105542+0000","last_became_peered":"2024-09-11T08:11:31.105542+0000","last_unstale":"2024-09-11T08:11:39.348485+0000","last_undegraded":"2024-09-11T08:11:39.348485+0000","last_fullsized":"2024-09-11T08:11:39.348485+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:06:17.897009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"67'12","reported_seq":39,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349960+0000","last_change":"2024-09-11T08:11:27.099017+0000","last_active":"2024-09-11T08:11:39.349960+0000","last_peered":"2024-09-11T08:11:39.349960+0000","last_clean":"2024-09-11T08:11:39.349960+0000","last_became_active":"2024-09-11T08:11:27.098583+0000","last_became_peered":"2024-09-11T08:11:27.098583+0000","last_unstale":"2024-09-11T08:11:39.349960+0000","last_undegraded":"2024-09-11T08:11:39.349960+0000","last_fullsized":"2024-09-11T08:11:39.349960+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:44:05.894625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349850+0000","last_change":"2024-09-11T08:11:25.088852+0000","last_active":"2024-09-11T08:11:39.349850+0000","last_peered":"2024-09-11T08:11:39.349850+0000","last_clean":"2024-09-11T08:11:39.349850+0000","last_became_active":"2024-09-11T08:11:25.088519+0000","last_became_peered":"2024-09-11T08:11:25.088519+0000","last_unstale":"2024-09-11T08:11:39.349850+0000","last_undegraded":"2024-09-11T08:11:39.349850+0000","last_fullsized":"2024-09-11T08:11:39.349850+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:14:17.016711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348656+0000","last_change":"2024-09-11T08:11:29.099409+0000","last_active":"2024-09-11T08:11:39.348656+0000","last_peered":"2024-09-11T08:11:39.348656+0000","last_clean":"2024-09-11T08:11:39.348656+0000","last_became_active":"2024-09-11T08:11:29.099131+0000","last_became_peered":"2024-09-11T08:11:29.099131+0000","last_unstale":"2024-09-11T08:11:39.348656+0000","last_undegraded":"2024-09-11T08:11:39.348656+0000","last_fullsized":"2024-09-11T08:11:39.348656+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:56:11.067512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799291+0000","last_change":"2024-09-11T08:11:31.110912+0000","last_active":"2024-09-11T08:11:38.799291+0000","last_peered":"2024-09-11T08:11:38.799291+0000","last_clean":"2024-09-11T08:11:38.799291+0000","last_became_active":"2024-09-11T08:11:31.110648+0000","last_became_peered":"2024-09-11T08:11:31.110648+0000","last_unstale":"2024-09-11T08:11:38.799291+0000","last_undegraded":"2024-09-11T08:11:38.799291+0000","last_fullsized":"2024-09-11T08:11:38.799291+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:55.345946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"67'15","reported_seq":41,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.797916+0000","last_change":"2024-09-11T08:11:27.086366+0000","last_active":"2024-09-11T08:11:38.797916+0000","last_peered":"2024-09-11T08:11:38.797916+0000","last_clean":"2024-09-11T08:11:38.797916+0000","last_became_active":"2024-09-11T08:11:27.086081+0000","last_became_peered":"2024-09-11T08:11:27.086081+0000","last_unstale":"2024-09-11T08:11:38.797916+0000","last_undegraded":"2024-09-11T08:11:38.797916+0000","last_fullsized":"2024-09-11T08:11:38.797916+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:53:26.600789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"60'2","reported_seq":35,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349654+0000","last_change":"2024-09-11T08:11:25.094547+0000","last_active":"2024-09-11T08:11:39.349654+0000","last_peered":"2024-09-11T08:11:39.349654+0000","last_clean":"2024-09-11T08:11:39.349654+0000","last_became_active":"2024-09-11T08:11:25.094277+0000","last_became_peered":"2024-09-11T08:11:25.094277+0000","last_unstale":"2024-09-11T08:11:39.349654+0000","last_undegraded":"2024-09-11T08:11:39.349654+0000","last_fullsized":"2024-09-11T08:11:39.349654+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:02:20.460779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":39,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:43.408600+0000","last_change":"2024-09-11T08:11:29.097168+0000","last_active":"2024-09-11T08:12:43.408600+0000","last_peered":"2024-09-11T08:12:43.408600+0000","last_clean":"2024-09-11T08:12:43.408600+0000","last_became_active":"2024-09-11T08:11:29.096734+0000","last_became_peered":"2024-09-11T08:11:29.096734+0000","last_unstale":"2024-09-11T08:12:43.408600+0000","last_undegraded":"2024-09-11T08:12:43.408600+0000","last_fullsized":"2024-09-11T08:12:43.408600+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:05:24.289993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798263+0000","last_change":"2024-09-11T08:11:31.418399+0000","last_active":"2024-09-11T08:11:38.798263+0000","last_peered":"2024-09-11T08:11:38.798263+0000","last_clean":"2024-09-11T08:11:38.798263+0000","last_became_active":"2024-09-11T08:11:31.418101+0000","last_became_peered":"2024-09-11T08:11:31.418101+0000","last_unstale":"2024-09-11T08:11:38.798263+0000","last_undegraded":"2024-09-11T08:11:38.798263+0000","last_fullsized":"2024-09-11T08:11:38.798263+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:01:10.314168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.808196+0000","last_change":"2024-09-11T08:11:25.094098+0000","last_active":"2024-09-11T08:11:38.808196+0000","last_peered":"2024-09-11T08:11:38.808196+0000","last_clean":"2024-09-11T08:11:38.808196+0000","last_became_active":"2024-09-11T08:11:25.093678+0000","last_became_peered":"2024-09-11T08:11:25.093678+0000","last_unstale":"2024-09-11T08:11:38.808196+0000","last_undegraded":"2024-09-11T08:11:38.808196+0000","last_fullsized":"2024-09-11T08:11:38.808196+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:42:39.518185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"67'6","reported_seq":25,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.800418+0000","last_change":"2024-09-11T08:11:27.090858+0000","last_active":"2024-09-11T08:11:38.800418+0000","last_peered":"2024-09-11T08:11:38.800418+0000","last_clean":"2024-09-11T08:11:38.800418+0000","last_became_active":"2024-09-11T08:11:27.090545+0000","last_became_peered":"2024-09-11T08:11:27.090545+0000","last_unstale":"2024-09-11T08:11:38.800418+0000","last_undegraded":"2024-09-11T08:11:38.800418+0000","last_fullsized":"2024-09-11T08:11:38.800418+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:41.233522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798344+0000","last_change":"2024-09-11T08:11:29.092988+0000","last_active":"2024-09-11T08:11:38.798344+0000","last_peered":"2024-09-11T08:11:38.798344+0000","last_clean":"2024-09-11T08:11:38.798344+0000","last_became_active":"2024-09-11T08:11:29.092698+0000","last_became_peered":"2024-09-11T08:11:29.092698+0000","last_unstale":"2024-09-11T08:11:38.798344+0000","last_undegraded":"2024-09-11T08:11:38.798344+0000","last_fullsized":"2024-09-11T08:11:38.798344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:37.407747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348994+0000","last_change":"2024-09-11T08:11:31.418227+0000","last_active":"2024-09-11T08:11:39.348994+0000","last_peered":"2024-09-11T08:11:39.348994+0000","last_clean":"2024-09-11T08:11:39.348994+0000","last_became_active":"2024-09-11T08:11:31.417918+0000","last_became_peered":"2024-09-11T08:11:31.417918+0000","last_unstale":"2024-09-11T08:11:39.348994+0000","last_undegraded":"2024-09-11T08:11:39.348994+0000","last_fullsized":"2024-09-11T08:11:39.348994+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:24:23.646738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"67'9","reported_seq":32,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799758+0000","last_change":"2024-09-11T08:11:27.093531+0000","last_active":"2024-09-11T08:11:38.799758+0000","last_peered":"2024-09-11T08:11:38.799758+0000","last_clean":"2024-09-11T08:11:38.799758+0000","last_became_active":"2024-09-11T08:11:27.093241+0000","last_became_peered":"2024-09-11T08:11:27.093241+0000","last_unstale":"2024-09-11T08:11:38.799758+0000","last_undegraded":"2024-09-11T08:11:38.799758+0000","last_fullsized":"2024-09-11T08:11:38.799758+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:58:48.283877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349188+0000","last_change":"2024-09-11T08:11:25.090713+0000","last_active":"2024-09-11T08:11:39.349188+0000","last_peered":"2024-09-11T08:11:39.349188+0000","last_clean":"2024-09-11T08:11:39.349188+0000","last_became_active":"2024-09-11T08:11:25.090416+0000","last_became_peered":"2024-09-11T08:11:25.090416+0000","last_unstale":"2024-09-11T08:11:39.349188+0000","last_undegraded":"2024-09-11T08:11:39.349188+0000","last_fullsized":"2024-09-11T08:11:39.349188+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:15.760643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.800273+0000","last_change":"2024-09-11T08:11:29.096594+0000","last_active":"2024-09-11T08:11:38.800273+0000","last_peered":"2024-09-11T08:11:38.800273+0000","last_clean":"2024-09-11T08:11:38.800273+0000","last_became_active":"2024-09-11T08:11:29.096267+0000","last_became_peered":"2024-09-11T08:11:29.096267+0000","last_unstale":"2024-09-11T08:11:38.800273+0000","last_undegraded":"2024-09-11T08:11:38.800273+0000","last_fullsized":"2024-09-11T08:11:38.800273+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:04:00.659569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349069+0000","last_change":"2024-09-11T08:11:31.107301+0000","last_active":"2024-09-11T08:11:39.349069+0000","last_peered":"2024-09-11T08:11:39.349069+0000","last_clean":"2024-09-11T08:11:39.349069+0000","last_became_active":"2024-09-11T08:11:31.106979+0000","last_became_peered":"2024-09-11T08:11:31.106979+0000","last_unstale":"2024-09-11T08:11:39.349069+0000","last_undegraded":"2024-09-11T08:11:39.349069+0000","last_fullsized":"2024-09-11T08:11:39.349069+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:32:14.830059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"67'9","reported_seq":32,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798542+0000","last_change":"2024-09-11T08:11:27.085114+0000","last_active":"2024-09-11T08:11:38.798542+0000","last_peered":"2024-09-11T08:11:38.798542+0000","last_clean":"2024-09-11T08:11:38.798542+0000","last_became_active":"2024-09-11T08:11:27.084820+0000","last_became_peered":"2024-09-11T08:11:27.084820+0000","last_unstale":"2024-09-11T08:11:38.798542+0000","last_undegraded":"2024-09-11T08:11:38.798542+0000","last_fullsized":"2024-09-11T08:11:38.798542+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:23.041169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799108+0000","last_change":"2024-09-11T08:11:25.092328+0000","last_active":"2024-09-11T08:11:38.799108+0000","last_peered":"2024-09-11T08:11:38.799108+0000","last_clean":"2024-09-11T08:11:38.799108+0000","last_became_active":"2024-09-11T08:11:25.091878+0000","last_became_peered":"2024-09-11T08:11:25.091878+0000","last_unstale":"2024-09-11T08:11:38.799108+0000","last_undegraded":"2024-09-11T08:11:38.799108+0000","last_fullsized":"2024-09-11T08:11:38.799108+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:02.395988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":39,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:43.408495+0000","last_change":"2024-09-11T08:11:29.091371+0000","last_active":"2024-09-11T08:12:43.408495+0000","last_peered":"2024-09-11T08:12:43.408495+0000","last_clean":"2024-09-11T08:12:43.408495+0000","last_became_active":"2024-09-11T08:11:29.091021+0000","last_became_peered":"2024-09-11T08:11:29.091021+0000","last_unstale":"2024-09-11T08:12:43.408495+0000","last_undegraded":"2024-09-11T08:12:43.408495+0000","last_fullsized":"2024-09-11T08:12:43.408495+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:03:11.692469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348602+0000","last_change":"2024-09-11T08:11:31.105834+0000","last_active":"2024-09-11T08:11:39.348602+0000","last_peered":"2024-09-11T08:11:39.348602+0000","last_clean":"2024-09-11T08:11:39.348602+0000","last_became_active":"2024-09-11T08:11:31.105487+0000","last_became_peered":"2024-09-11T08:11:31.105487+0000","last_unstale":"2024-09-11T08:11:39.348602+0000","last_undegraded":"2024-09-11T08:11:39.348602+0000","last_fullsized":"2024-09-11T08:11:39.348602+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:49:24.252947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"67'10","reported_seq":31,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.800049+0000","last_change":"2024-09-11T08:11:27.087770+0000","last_active":"2024-09-11T08:11:38.800049+0000","last_peered":"2024-09-11T08:11:38.800049+0000","last_clean":"2024-09-11T08:11:38.800049+0000","last_became_active":"2024-09-11T08:11:27.086652+0000","last_became_peered":"2024-09-11T08:11:27.086652+0000","last_unstale":"2024-09-11T08:11:38.800049+0000","last_undegraded":"2024-09-11T08:11:38.800049+0000","last_fullsized":"2024-09-11T08:11:38.800049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:39:59.515088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349248+0000","last_change":"2024-09-11T08:11:25.090560+0000","last_active":"2024-09-11T08:11:39.349248+0000","last_peered":"2024-09-11T08:11:39.349248+0000","last_clean":"2024-09-11T08:11:39.349248+0000","last_became_active":"2024-09-11T08:11:25.090195+0000","last_became_peered":"2024-09-11T08:11:25.090195+0000","last_unstale":"2024-09-11T08:11:39.349248+0000","last_undegraded":"2024-09-11T08:11:39.349248+0000","last_fullsized":"2024-09-11T08:11:39.349248+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:52.830520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":39,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T08:12:43.408589+0000","last_change":"2024-09-11T08:11:29.099539+0000","last_active":"2024-09-11T08:12:43.408589+0000","last_peered":"2024-09-11T08:12:43.408589+0000","last_clean":"2024-09-11T08:12:43.408589+0000","last_became_active":"2024-09-11T08:11:29.099258+0000","last_became_peered":"2024-09-11T08:11:29.099258+0000","last_unstale":"2024-09-11T08:12:43.408589+0000","last_undegraded":"2024-09-11T08:12:43.408589+0000","last_fullsized":"2024-09-11T08:12:43.408589+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:57:12.119930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799362+0000","last_change":"2024-09-11T08:11:31.113611+0000","last_active":"2024-09-11T08:11:38.799362+0000","last_peered":"2024-09-11T08:11:38.799362+0000","last_clean":"2024-09-11T08:11:38.799362+0000","last_became_active":"2024-09-11T08:11:31.113314+0000","last_became_peered":"2024-09-11T08:11:31.113314+0000","last_unstale":"2024-09-11T08:11:38.799362+0000","last_undegraded":"2024-09-11T08:11:38.799362+0000","last_fullsized":"2024-09-11T08:11:38.799362+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:42:47.265588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":35,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349016+0000","last_change":"2024-09-11T08:11:27.099097+0000","last_active":"2024-09-11T08:11:39.349016+0000","last_peered":"2024-09-11T08:11:39.349016+0000","last_clean":"2024-09-11T08:11:39.349016+0000","last_became_active":"2024-09-11T08:11:27.098659+0000","last_became_peered":"2024-09-11T08:11:27.098659+0000","last_unstale":"2024-09-11T08:11:39.349016+0000","last_undegraded":"2024-09-11T08:11:39.349016+0000","last_fullsized":"2024-09-11T08:11:39.349016+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:05:53.276172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349190+0000","last_change":"2024-09-11T08:11:25.091639+0000","last_active":"2024-09-11T08:11:39.349190+0000","last_peered":"2024-09-11T08:11:39.349190+0000","last_clean":"2024-09-11T08:11:39.349190+0000","last_became_active":"2024-09-11T08:11:25.091358+0000","last_became_peered":"2024-09-11T08:11:25.091358+0000","last_unstale":"2024-09-11T08:11:39.349190+0000","last_undegraded":"2024-09-11T08:11:39.349190+0000","last_fullsized":"2024-09-11T08:11:39.349190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:44.266849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.847978+0000","last_change":"2024-09-11T08:11:29.094531+0000","last_active":"2024-09-11T08:11:38.847978+0000","last_peered":"2024-09-11T08:11:38.847978+0000","last_clean":"2024-09-11T08:11:38.847978+0000","last_became_active":"2024-09-11T08:11:29.094244+0000","last_became_peered":"2024-09-11T08:11:29.094244+0000","last_unstale":"2024-09-11T08:11:38.847978+0000","last_undegraded":"2024-09-11T08:11:38.847978+0000","last_fullsized":"2024-09-11T08:11:38.847978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:35.475093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799518+0000","last_change":"2024-09-11T08:11:31.110223+0000","last_active":"2024-09-11T08:11:38.799518+0000","last_peered":"2024-09-11T08:11:38.799518+0000","last_clean":"2024-09-11T08:11:38.799518+0000","last_became_active":"2024-09-11T08:11:31.109929+0000","last_became_peered":"2024-09-11T08:11:31.109929+0000","last_unstale":"2024-09-11T08:11:38.799518+0000","last_undegraded":"2024-09-11T08:11:38.799518+0000","last_fullsized":"2024-09-11T08:11:38.799518+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:46:50.447850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":32,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.848206+0000","last_change":"2024-09-11T08:11:27.095479+0000","last_active":"2024-09-11T08:11:38.848206+0000","last_peered":"2024-09-11T08:11:38.848206+0000","last_clean":"2024-09-11T08:11:38.848206+0000","last_became_active":"2024-09-11T08:11:27.091860+0000","last_became_peered":"2024-09-11T08:11:27.091860+0000","last_unstale":"2024-09-11T08:11:38.848206+0000","last_undegraded":"2024-09-11T08:11:38.848206+0000","last_fullsized":"2024-09-11T08:11:38.848206+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:14.627903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.349254+0000","last_change":"2024-09-11T08:11:25.089770+0000","last_active":"2024-09-11T08:11:39.349254+0000","last_peered":"2024-09-11T08:11:39.349254+0000","last_clean":"2024-09-11T08:11:39.349254+0000","last_became_active":"2024-09-11T08:11:25.089471+0000","last_became_peered":"2024-09-11T08:11:25.089471+0000","last_unstale":"2024-09-11T08:11:39.349254+0000","last_undegraded":"2024-09-11T08:11:39.349254+0000","last_fullsized":"2024-09-11T08:11:39.349254+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:10.414780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799364+0000","last_change":"2024-09-11T08:11:29.099521+0000","last_active":"2024-09-11T08:11:38.799364+0000","last_peered":"2024-09-11T08:11:38.799364+0000","last_clean":"2024-09-11T08:11:38.799364+0000","last_became_active":"2024-09-11T08:11:29.099072+0000","last_became_peered":"2024-09-11T08:11:29.099072+0000","last_unstale":"2024-09-11T08:11:38.799364+0000","last_undegraded":"2024-09-11T08:11:38.799364+0000","last_fullsized":"2024-09-11T08:11:38.799364+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:43.207670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798370+0000","last_change":"2024-09-11T08:11:31.108660+0000","last_active":"2024-09-11T08:11:38.798370+0000","last_peered":"2024-09-11T08:11:38.798370+0000","last_clean":"2024-09-11T08:11:38.798370+0000","last_became_active":"2024-09-11T08:11:31.108359+0000","last_became_peered":"2024-09-11T08:11:31.108359+0000","last_unstale":"2024-09-11T08:11:38.798370+0000","last_undegraded":"2024-09-11T08:11:38.798370+0000","last_fullsized":"2024-09-11T08:11:38.798370+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:34.391986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"67'11","reported_seq":35,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798637+0000","last_change":"2024-09-11T08:11:27.085607+0000","last_active":"2024-09-11T08:11:38.798637+0000","last_peered":"2024-09-11T08:11:38.798637+0000","last_clean":"2024-09-11T08:11:38.798637+0000","last_became_active":"2024-09-11T08:11:27.085326+0000","last_became_peered":"2024-09-11T08:11:27.085326+0000","last_unstale":"2024-09-11T08:11:38.798637+0000","last_undegraded":"2024-09-11T08:11:38.798637+0000","last_fullsized":"2024-09-11T08:11:38.798637+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:32:39.314248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.798380+0000","last_change":"2024-09-11T08:11:25.097786+0000","last_active":"2024-09-11T08:11:38.798380+0000","last_peered":"2024-09-11T08:11:38.798380+0000","last_clean":"2024-09-11T08:11:38.798380+0000","last_became_active":"2024-09-11T08:11:25.097485+0000","last_became_peered":"2024-09-11T08:11:25.097485+0000","last_unstale":"2024-09-11T08:11:38.798380+0000","last_undegraded":"2024-09-11T08:11:38.798380+0000","last_fullsized":"2024-09-11T08:11:38.798380+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:28.843596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348509+0000","last_change":"2024-09-11T08:11:29.097789+0000","last_active":"2024-09-11T08:11:39.348509+0000","last_peered":"2024-09-11T08:11:39.348509+0000","last_clean":"2024-09-11T08:11:39.348509+0000","last_became_active":"2024-09-11T08:11:29.097518+0000","last_became_peered":"2024-09-11T08:11:29.097518+0000","last_unstale":"2024-09-11T08:11:39.348509+0000","last_undegraded":"2024-09-11T08:11:39.348509+0000","last_fullsized":"2024-09-11T08:11:39.348509+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:44.929779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799522+0000","last_change":"2024-09-11T08:11:31.417458+0000","last_active":"2024-09-11T08:11:38.799522+0000","last_peered":"2024-09-11T08:11:38.799522+0000","last_clean":"2024-09-11T08:11:38.799522+0000","last_became_active":"2024-09-11T08:11:31.416299+0000","last_became_peered":"2024-09-11T08:11:31.416299+0000","last_unstale":"2024-09-11T08:11:38.799522+0000","last_undegraded":"2024-09-11T08:11:38.799522+0000","last_fullsized":"2024-09-11T08:11:38.799522+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:13.575094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":22,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799105+0000","last_change":"2024-09-11T08:11:27.089964+0000","last_active":"2024-09-11T08:11:38.799105+0000","last_peered":"2024-09-11T08:11:38.799105+0000","last_clean":"2024-09-11T08:11:38.799105+0000","last_became_active":"2024-09-11T08:11:27.086636+0000","last_became_peered":"2024-09-11T08:11:27.086636+0000","last_unstale":"2024-09-11T08:11:38.799105+0000","last_undegraded":"2024-09-11T08:11:38.799105+0000","last_fullsized":"2024-09-11T08:11:38.799105+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:32.772749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799178+0000","last_change":"2024-09-11T08:11:25.100896+0000","last_active":"2024-09-11T08:11:38.799178+0000","last_peered":"2024-09-11T08:11:38.799178+0000","last_clean":"2024-09-11T08:11:38.799178+0000","last_became_active":"2024-09-11T08:11:25.100298+0000","last_became_peered":"2024-09-11T08:11:25.100298+0000","last_unstale":"2024-09-11T08:11:38.799178+0000","last_undegraded":"2024-09-11T08:11:38.799178+0000","last_fullsized":"2024-09-11T08:11:38.799178+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:57:41.021067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.808217+0000","last_change":"2024-09-11T08:11:29.098400+0000","last_active":"2024-09-11T08:11:38.808217+0000","last_peered":"2024-09-11T08:11:38.808217+0000","last_clean":"2024-09-11T08:11:38.808217+0000","last_became_active":"2024-09-11T08:11:29.098104+0000","last_became_peered":"2024-09-11T08:11:29.098104+0000","last_unstale":"2024-09-11T08:11:38.808217+0000","last_undegraded":"2024-09-11T08:11:38.808217+0000","last_fullsized":"2024-09-11T08:11:38.808217+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:08:10.525216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:39.348612+0000","last_change":"2024-09-11T08:11:31.109269+0000","last_active":"2024-09-11T08:11:39.348612+0000","last_peered":"2024-09-11T08:11:39.348612+0000","last_clean":"2024-09-11T08:11:39.348612+0000","last_became_active":"2024-09-11T08:11:31.108848+0000","last_became_peered":"2024-09-11T08:11:31.108848+0000","last_unstale":"2024-09-11T08:11:39.348612+0000","last_undegraded":"2024-09-11T08:11:39.348612+0000","last_fullsized":"2024-09-11T08:11:39.348612+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:17.843343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.848093+0000","last_change":"2024-09-11T08:11:31.104544+0000","last_active":"2024-09-11T08:11:38.848093+0000","last_peered":"2024-09-11T08:11:38.848093+0000","last_clean":"2024-09-11T08:11:38.848093+0000","last_became_active":"2024-09-11T08:11:31.104169+0000","last_became_peered":"2024-09-11T08:11:31.104169+0000","last_unstale":"2024-09-11T08:11:38.848093+0000","last_undegraded":"2024-09-11T08:11:38.848093+0000","last_fullsized":"2024-09-11T08:11:38.848093+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:30.086104+0000","last_clean_scrub_stamp":"2024-09-11T08:11:30.086104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:02.802258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799322+0000","last_change":"2024-09-11T08:11:25.096405+0000","last_active":"2024-09-11T08:11:38.799322+0000","last_peered":"2024-09-11T08:11:38.799322+0000","last_clean":"2024-09-11T08:11:38.799322+0000","last_became_active":"2024-09-11T08:11:25.095993+0000","last_became_peered":"2024-09-11T08:11:25.095993+0000","last_unstale":"2024-09-11T08:11:38.799322+0000","last_undegraded":"2024-09-11T08:11:38.799322+0000","last_fullsized":"2024-09-11T08:11:38.799322+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:24.067143+0000","last_clean_scrub_stamp":"2024-09-11T08:11:24.067143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:46.111533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"67'11","reported_seq":35,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.808616+0000","last_change":"2024-09-11T08:11:27.089123+0000","last_active":"2024-09-11T08:11:38.808616+0000","last_peered":"2024-09-11T08:11:38.808616+0000","last_clean":"2024-09-11T08:11:38.808616+0000","last_became_active":"2024-09-11T08:11:27.088855+0000","last_became_peered":"2024-09-11T08:11:27.088855+0000","last_unstale":"2024-09-11T08:11:38.808616+0000","last_undegraded":"2024-09-11T08:11:38.808616+0000","last_fullsized":"2024-09-11T08:11:38.808616+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:26.073608+0000","last_clean_scrub_stamp":"2024-09-11T08:11:26.073608+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:14:01.493971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T08:11:38.799435+0000","last_change":"2024-09-11T08:11:29.100530+0000","last_active":"2024-09-11T08:11:38.799435+0000","last_peered":"2024-09-11T08:11:38.799435+0000","last_clean":"2024-09-11T08:11:38.799435+0000","last_became_active":"2024-09-11T08:11:29.100218+0000","last_became_peered":"2024-09-11T08:11:29.100218+0000","last_unstale":"2024-09-11T08:11:38.799435+0000","last_undegraded":"2024-09-11T08:11:38.799435+0000","last_fullsized":"2024-09-11T08:11:38.799435+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:11:28.076468+0000","last_clean_scrub_stamp":"2024-09-11T08:11:28.076468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:25.054678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":75,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":258,"num_write":179,"num_write_kb":3194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234007,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298688,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442368,"statfs":{"total":95990841344,"available":95684984832,"internally_reserved":0,"allocated":1363968,"data_stored":907109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.623}]},{"osd":1,"last update":"Wed Sep 11 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.65900000000000003}]},{"osd":2,"last update":"Wed Sep 11 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.64200000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.70999999999999996}]},{"osd":4,"last update":"Wed Sep 11 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.41199999999999998}]},{"osd":5,"last update":"Wed Sep 11 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.45500000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.51200000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495644,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298660,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442396,"statfs":{"total":95990841344,"available":95685013504,"internally_reserved":0,"allocated":1335296,"data_stored":905819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:12:48 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.378,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72399999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51400000000000001,"15min":0.51300000000000001},"min":{"1min":0.41199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.66300000000000003,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.57999999999999996}]},{"osd":1,"last update":"Wed Sep 11 08:12:48 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51800000000000002,"15min":0.52100000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64600000000000002,"5min":0.753,"15min":0.753},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.48799999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:12:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50700000000000001,"15min":0.505},"min":{"1min":0.377,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.505},{"interface":"front","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.35999999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.63200000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.63500000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:12:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69399999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40899999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.53000000000000003}]},{"osd":4,"last update":"Wed Sep 11 08:12:48 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.39100000000000001,"15min":0.38900000000000001},"min":{"1min":0.29399999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.41199999999999998,"15min":0.40400000000000003},"min":{"1min":0.34000000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.61399999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.38500000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:12:48 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.42299999999999999,"15min":0.41499999999999998},"min":{"1min":0.33100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.434},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.41099999999999998,"15min":0.40200000000000002},"min":{"1min":0.311,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.34799999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:12:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51900000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757279,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298092,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442964,"statfs":{"total":95990841344,"available":95685595136,"internally_reserved":0,"allocated":753664,"data_stored":316015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:12:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.52300000000000002,"15min":0.53000000000000003},"min":{"1min":0.34300000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.69899999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.51000000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:12:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52700000000000002,"15min":0.52800000000000002},"min":{"1min":0.34599999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.73799999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.622},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.55100000000000005,"15min":0.56299999999999994},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.745,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:12:30 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.54000000000000004,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.39900000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66000000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.58499999999999996}]},{"osd":3,"last update":"Wed Sep 11 08:12:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.33500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68400000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.495},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55800000000000005}]},{"osd":4,"last update":"Wed Sep 11 08:12:30 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.67200000000000004,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.438},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.309,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.51600000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.39600000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:12:49 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42399999999999999,"15min":0.42599999999999999},"min":{"1min":0.27600000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.59699999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.64100000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46400000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:12:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.53200000000000003}]}]},{"osd":4,"up_from":30,"seq":128849018917,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298108,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442948,"statfs":{"total":95990841344,"available":95685578752,"internally_reserved":0,"allocated":770048,"data_stored":317103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:12:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.51500000000000001,"15min":0.50800000000000001},"min":{"1min":0.42599999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.39200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed Sep 11 08:12:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.441},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.56699999999999995}]},{"osd":2,"last update":"Wed Sep 11 08:12:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.42299999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.5},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.47699999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:12:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.378},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52300000000000002,"15min":0.51800000000000002},"min":{"1min":0.41699999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.40200000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:12:26 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.375,"15min":0.36899999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.375,"15min":0.36599999999999999},"min":{"1min":0.31900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.63}]},{"osd":6,"last update":"Wed Sep 11 08:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.66800000000000004}]},{"osd":7,"last update":"Wed Sep 11 08:12:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.53200000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182442,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298116,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442940,"statfs":{"total":95990841344,"available":95685570560,"internally_reserved":0,"allocated":778240,"data_stored":316522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:12:45 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.34100000000000003,"15min":0.34499999999999997},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.41999999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.17499999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.47799999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.26800000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:12:45 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.376,"15min":0.38},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.51500000000000001,"5min":0.626,"15min":0.626},"last":0.187},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.40799999999999997,"15min":0.42799999999999999},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48099999999999998,"5min":0.751,"15min":0.751},"last":0.24199999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:12:45 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40999999999999998,"15min":0.41999999999999998},"min":{"1min":0.29399999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.49399999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.39300000000000002,"15min":0.40500000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.46600000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.23100000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:12:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52800000000000002,"15min":0.53200000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.46500000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:12:27 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.435},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.53500000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.377}]},{"osd":6,"last update":"Wed Sep 11 08:11:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.48199999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:12:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.499},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.44500000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298080,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442976,"statfs":{"total":95990841344,"available":95685607424,"internally_reserved":0,"allocated":741376,"data_stored":315143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:12:16 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36199999999999999,"15min":0.35399999999999998},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.60199999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.36499999999999999,"15min":0.36299999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.52300000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.38700000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:12:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.38100000000000001,"15min":0.377},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.54500000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58099999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.439}]},{"osd":3,"last update":"Wed Sep 11 08:12:46 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.42899999999999999,"15min":0.434},"min":{"1min":0.28799999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.57699999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.376},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.442,"15min":0.44700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63700000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:12:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55200000000000005,"15min":0.55600000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69899999999999995,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68799999999999994,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.65500000000000003}]},{"osd":5,"last update":"Wed Sep 11 08:12:30 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71999999999999997,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.66600000000000004}]},{"osd":6,"last update":"Wed Sep 11 08:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Wed Sep 11 08:12:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.623}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298148,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442908,"statfs":{"total":95990841344,"available":95685537792,"internally_reserved":0,"allocated":811008,"data_stored":318606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.34300000000000003,"15min":0.32300000000000001},"min":{"1min":0.224,"5min":0.221,"15min":0.221},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.34799999999999998},"min":{"1min":0.255,"5min":0.214,"15min":0.214},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.36399999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:12:18 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36099999999999999,"15min":0.35099999999999998},"min":{"1min":0.23599999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.54500000000000004,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.311},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40699999999999997,"15min":0.40300000000000002},"min":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.25600000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:12:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.45700000000000002,"15min":0.46300000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.622,"5min":0.747,"15min":0.747},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.65500000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.33900000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:12:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55200000000000005,"15min":0.54300000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:12:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.50600000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.495},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.441}]},{"osd":7,"last update":"Wed Sep 11 08:12:13 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.46000000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298684,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442372,"statfs":{"total":95990841344,"available":95684988928,"internally_reserved":0,"allocated":1363968,"data_stored":907667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.32700000000000001,"15min":0.317},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.434,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.316},"min":{"1min":0.22900000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.36399999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:12:22 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.36099999999999999,"15min":0.36499999999999999},"min":{"1min":0.27200000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.38100000000000001},"min":{"1min":0.26600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.49299999999999999,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.33300000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:12:43 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40600000000000003,"15min":0.41199999999999998},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.55700000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.317},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.41799999999999998,"15min":0.42299999999999999},"min":{"1min":0.28899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.498,"5min":0.62,"15min":0.62},"last":0.28599999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:12:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.502,"15min":0.496},"min":{"1min":0.436,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.65700000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.48099999999999998,"15min":0.48199999999999998},"min":{"1min":0.35999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.57599999999999996}]},{"osd":5,"last update":"Wed Sep 11 08:12:27 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.39600000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed Sep 11 08:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.55300000000000005}]},{"osd":7,"last update":"Wed Sep 11 08:12:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.65600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:12:56.094 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T08:12:56.094 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T08:12:56.094 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T08:12:56.095 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T08:12:56.372 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:12:56.805 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:56 smithi071 ceph-mon[39897]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:12:56.805 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:56 smithi071 ceph-mon[39897]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:56.805 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:56 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:56.805 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:56 smithi071 ceph-mon[25834]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:12:56.805 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:56 smithi071 ceph-mon[25834]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:56.806 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:56 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:56.931 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:12:56 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: ::ffff:172.21.15.196 - - [11/Sep/2024:08:12:56] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:56.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:56 smithi196 ceph-mon[30677]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:12:56.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:56 smithi196 ceph-mon[30677]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:12:56.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:56 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:12:58.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:58 smithi196 ceph-mon[30677]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:58.932 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T08:12:58.932 INFO:teuthology.orchestra.run.smithi071.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T08:12:58.933 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:58 smithi071 ceph-mon[25834]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:58.933 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:58 smithi071 ceph-mon[39897]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:00.155 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:00.156 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:00.156 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[39897]: from='client.? 172.21.15.71:0/2962125853' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:13:00.156 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[39897]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:00.156 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:00.156 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:00.156 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:00.156 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:00.156 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:00.156 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:00.157 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:00.157 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[25834]: from='client.? 172.21.15.71:0/2962125853' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:13:00.157 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[25834]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:00.157 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:00.157 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:00.157 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:00.157 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:00.157 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:12:59 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:00.178 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T08:13:00.178 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T08:13:00.178 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T08:13:00.188 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi071.front.sepia.ceph.com 2024-09-11T08:13:00.189 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-09-11T08:13:00.189 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:59 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:00.189 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:59 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:00.189 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:59 smithi196 ceph-mon[30677]: from='client.? 172.21.15.71:0/2962125853' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:13:00.190 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:59 smithi196 ceph-mon[30677]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:00.190 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:59 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:00.190 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:59 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:00.190 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:59 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:00.190 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:59 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:00.190 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:12:59 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:00.935 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:00 smithi071 ceph-mon[25834]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:13:00.935 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:00 smithi071 ceph-mon[25834]: Reconfiguring daemon alertmanager.a on smithi071 2024-09-11T08:13:00.936 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:00 smithi071 ceph-mon[39897]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:13:00.936 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:00 smithi071 ceph-mon[39897]: Reconfiguring daemon alertmanager.a on smithi071 2024-09-11T08:13:01.180 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:13:01.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:00 smithi196 ceph-mon[30677]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:13:01.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:00 smithi196 ceph-mon[30677]: Reconfiguring daemon alertmanager.a on smithi071 2024-09-11T08:13:01.912 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-11T08:13:01.912 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-11T08:13:01.912 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:01 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-11T08:13:01.912 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-09-11T08:13:01.913 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:01 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-11T08:13:01.913 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:01 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-09-11T08:13:01.913 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:01 smithi071 ceph-mon[39897]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:01.913 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:01 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:01.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-11T08:13:01.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-11T08:13:01.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:01 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-11T08:13:01.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-09-11T08:13:01.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:01 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-11T08:13:01.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:01 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-09-11T08:13:01.914 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:01 smithi071 ceph-mon[25834]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:01.914 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:01 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:02.124 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-11T08:13:02.124 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-11T08:13:02.124 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:01 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-11T08:13:02.125 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-09-11T08:13:02.125 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:01 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-11T08:13:02.125 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:01 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-09-11T08:13:02.125 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:01 smithi196 ceph-mon[30677]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:02.125 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:01 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:02.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:13:02 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug there is no tcmu-runner data available 2024-09-11T08:13:02.924 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:02 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-11T08:13:02.925 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:02 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-09-11T08:13:02.925 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:02 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-09-11T08:13:02.925 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:02 smithi071 ceph-mon[39897]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T08:13:02.925 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:02 smithi071 ceph-mon[39897]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:02.925 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:02 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-11T08:13:02.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:02 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-09-11T08:13:02.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:02 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-09-11T08:13:02.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:02 smithi071 ceph-mon[25834]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T08:13:02.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:02 smithi071 ceph-mon[25834]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:03.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:02 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-11T08:13:03.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:02 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-09-11T08:13:03.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:02 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-09-11T08:13:03.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:02 smithi196 ceph-mon[30677]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T08:13:03.182 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:02 smithi196 ceph-mon[30677]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:04.188 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:03 smithi196 ceph-mon[30677]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T08:13:04.189 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:03 smithi196 ceph-mon[30677]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:04.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:03 smithi071 ceph-mon[25834]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T08:13:04.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:03 smithi071 ceph-mon[25834]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:04.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:03 smithi071 ceph-mon[39897]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T08:13:04.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:03 smithi071 ceph-mon[39897]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:04.527 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:04 smithi071 systemd[1]: Stopping Ceph alertmanager.a for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:13:04.763 INFO:teuthology.orchestra.run.smithi071.stdout:167 167 2024-09-11T08:13:04.944 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:04 smithi071 ceph-mon[25834]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-11T08:13:04.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:04 smithi071 ceph-mon[39897]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-11T08:13:05.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:04 smithi196 ceph-mon[30677]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-11T08:13:05.516 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-11T08:13:05.758 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:05 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a[83304]: ts=2024-09-11T08:13:05.672Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:13:05.758 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:05 smithi071 podman[86414]: 2024-09-11 08:13:05.693530457 +0000 UTC m=+0.663352531 container died fa7cd8a700e4a553960fb20a7e3fd98ce209819e893dfe4f79ca3d287bd9f985 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:13:06.017 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:05 smithi071 ceph-mon[25834]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:06.018 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:05 smithi071 ceph-mon[39897]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:06.018 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:06 smithi071 podman[86414]: 2024-09-11 08:13:06.003340158 +0000 UTC m=+0.973162228 container cleanup fa7cd8a700e4a553960fb20a7e3fd98ce209819e893dfe4f79ca3d287bd9f985 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:13:06.018 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:06 smithi071 bash[86414]: ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T08:13:06.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:05 smithi196 ceph-mon[30677]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:13:06.552 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:06 smithi071 podman[86460]: 2024-09-11 08:13:06.229672032 +0000 UTC m=+0.532615621 container remove fa7cd8a700e4a553960fb20a7e3fd98ce209819e893dfe4f79ca3d287bd9f985 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:13:06.552 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:06 smithi071 podman[86460]: 2024-09-11 08:13:06.305210591 +0000 UTC m=+0.608154184 volume remove fc85b30706bebcc56a0ca7dc06dde08dceb4b9b36e0c5beb070a42378d094ecc 2024-09-11T08:13:06.931 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:13:06 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: ::ffff:172.21.15.196 - - [11/Sep/2024:08:13:06] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:13:07.112 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:06 smithi071 podman[86511]: 2024-09-11 08:13:06.90873493 +0000 UTC m=+0.545141101 container died 301add327bb2d90797e90b31a5bbf19bc50b10ef744a3edae0044449d7d3424c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=affectionate_bell, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-11T08:13:07.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:07 smithi071 ceph-mon[25834]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-11T08:13:07.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:07 smithi071 ceph-mon[25834]: Cluster is now healthy 2024-09-11T08:13:07.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:07 smithi071 ceph-mon[39897]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-11T08:13:07.445 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:07 smithi071 ceph-mon[39897]: Cluster is now healthy 2024-09-11T08:13:07.446 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:07 smithi071 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T08:13:07.446 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:07 smithi071 systemd[1]: Stopped Ceph alertmanager.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:13:07.446 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:07 smithi071 systemd[1]: Starting Ceph alertmanager.a for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:13:07.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:07 smithi196 ceph-mon[30677]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-11T08:13:07.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:07 smithi196 ceph-mon[30677]: Cluster is now healthy 2024-09-11T08:13:08.185 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:07 smithi071 podman[86671]: 2024-09-11 08:13:07.861475566 +0000 UTC m=+0.016820814 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-11T08:13:08.185 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:07 smithi071 podman[86671]: 2024-09-11 08:13:07.965089766 +0000 UTC m=+0.120434998 volume create 8a0c177b444b04609e8ba923dcb79997bc56fc9c336df31491394e9e94282581 2024-09-11T08:13:08.185 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:08 smithi071 podman[86671]: 2024-09-11 08:13:08.065531602 +0000 UTC m=+0.220876875 container create c42e8261ece503a62f3df687fcb3709bdeead04e94bb1fe7f23255f026d89e87 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:13:08.438 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:08 smithi071 ceph-mon[39897]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 2 objects/s recovering 2024-09-11T08:13:08.438 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:08 smithi071 ceph-mon[25834]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 2 objects/s recovering 2024-09-11T08:13:08.439 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:08 smithi071 podman[86671]: 2024-09-11 08:13:08.395277692 +0000 UTC m=+0.550622925 container init c42e8261ece503a62f3df687fcb3709bdeead04e94bb1fe7f23255f026d89e87 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:13:08.439 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:08 smithi071 podman[86671]: 2024-09-11 08:13:08.40048208 +0000 UTC m=+0.555827314 container start c42e8261ece503a62f3df687fcb3709bdeead04e94bb1fe7f23255f026d89e87 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:13:08.439 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:08 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a[86704]: ts=2024-09-11T08:13:08.413Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-11T08:13:08.439 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:08 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a[86704]: ts=2024-09-11T08:13:08.413Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-11T08:13:08.439 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:08 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a[86704]: ts=2024-09-11T08:13:08.414Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.71 port=9094 2024-09-11T08:13:08.439 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:08 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a[86704]: ts=2024-09-11T08:13:08.415Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-11T08:13:08.439 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:08 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a[86704]: ts=2024-09-11T08:13:08.438Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:13:08.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:08 smithi196 ceph-mon[30677]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 2 objects/s recovering 2024-09-11T08:13:08.694 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:08 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a[86704]: ts=2024-09-11T08:13:08.438Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:13:08.695 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:08 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a[86704]: ts=2024-09-11T08:13:08.441Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-11T08:13:08.695 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:08 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a[86704]: ts=2024-09-11T08:13:08.441Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-11T08:13:08.695 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:08 smithi071 bash[86671]: c42e8261ece503a62f3df687fcb3709bdeead04e94bb1fe7f23255f026d89e87 2024-09-11T08:13:08.695 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:08 smithi071 systemd[1]: Started Ceph alertmanager.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:13:08.880 INFO:teuthology.orchestra.run.smithi071.stdout:Backend: cephadm 2024-09-11T08:13:08.881 INFO:teuthology.orchestra.run.smithi071.stdout:Available: Yes 2024-09-11T08:13:08.881 INFO:teuthology.orchestra.run.smithi071.stdout:Paused: No 2024-09-11T08:13:09.534 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:09 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:09.534 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:09 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:09.534 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:09 smithi071 ceph-mon[39897]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:13:09.534 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:09 smithi071 ceph-mon[39897]: Reconfiguring daemon prometheus.a on smithi196 2024-09-11T08:13:09.534 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:09 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:09.534 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:09 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:09.534 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:09 smithi071 ceph-mon[25834]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:13:09.534 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:09 smithi071 ceph-mon[25834]: Reconfiguring daemon prometheus.a on smithi196 2024-09-11T08:13:09.558 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-11T08:13:09.801 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:13:09.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:09 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:09.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:09 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:09.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:09 smithi196 ceph-mon[30677]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:13:09.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:09 smithi196 ceph-mon[30677]: Reconfiguring daemon prometheus.a on smithi196 2024-09-11T08:13:10.534 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:10 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a[86704]: ts=2024-09-11T08:13:10.415Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000474269s 2024-09-11T08:13:10.795 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:10 smithi196 ceph-mon[30677]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:10.795 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:10 smithi196 ceph-mon[30677]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 2 objects/s recovering 2024-09-11T08:13:10.861 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:10 smithi071 ceph-mon[39897]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:10.861 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:10 smithi071 ceph-mon[39897]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 2 objects/s recovering 2024-09-11T08:13:10.862 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:10 smithi071 ceph-mon[25834]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:10.862 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:10 smithi071 ceph-mon[25834]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 2 objects/s recovering 2024-09-11T08:13:11.588 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-11T08:13:11.589 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.a smithi071 *:9093,9094 starting - - - - 2024-09-11T08:13:11.589 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.a smithi196 *:3000 running (28s) 12s ago 27s 79.8M - 9.4.7 954c08fa6188 b20068dfb13a 2024-09-11T08:13:11.589 INFO:teuthology.orchestra.run.smithi071.stdout:iscsi.iscsi.a smithi196 *:5000 running (94s) 12s ago 94s 58.7M - 3.8 4a918172a497 da81cc55fa16 2024-09-11T08:13:11.589 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.x smithi196 *:8443 running (5m) 12s ago 5m 398M - 17.2.7-1623-gd0e6828a 4a918172a497 2cc3e7f0e1d3 2024-09-11T08:13:11.589 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.y smithi071 *:9283,8765 running (6m) 11s ago 6m 444M - 17.2.7-1623-gd0e6828a 4a918172a497 da69cb3a1c35 2024-09-11T08:13:11.589 INFO:teuthology.orchestra.run.smithi071.stdout:mon.a smithi071 running (6m) 11s ago 7m 47.8M 2048M 17.2.7-1623-gd0e6828a 4a918172a497 e6731b5b742d 2024-09-11T08:13:11.589 INFO:teuthology.orchestra.run.smithi071.stdout:mon.b smithi196 running (5m) 12s ago 5m 40.2M 2048M 17.2.7-1623-gd0e6828a 4a918172a497 218d41aee37f 2024-09-11T08:13:11.589 INFO:teuthology.orchestra.run.smithi071.stdout:mon.c smithi071 running (5m) 11s ago 5m 39.0M 2048M 17.2.7-1623-gd0e6828a 4a918172a497 e22f17c69619 2024-09-11T08:13:11.589 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.a smithi071 *:9100 running (21s) 11s ago 21s 5448k - 1.5.0 0da6a335fe13 4837cdf81df5 2024-09-11T08:13:11.589 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.b smithi196 *:9100 running (17s) 12s ago 17s 3590k - 1.5.0 0da6a335fe13 214d90a9f27e 2024-09-11T08:13:11.589 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (4m) 11s ago 4m 48.7M 3760M 17.2.7-1623-gd0e6828a 4a918172a497 e3a76e1084fd 2024-09-11T08:13:11.590 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (4m) 11s ago 4m 50.3M 3760M 17.2.7-1623-gd0e6828a 4a918172a497 8d8386f10e1f 2024-09-11T08:13:11.590 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (4m) 11s ago 4m 46.3M 3760M 17.2.7-1623-gd0e6828a 4a918172a497 88e63e05de7d 2024-09-11T08:13:11.590 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi071 running (3m) 11s ago 3m 43.2M 3760M 17.2.7-1623-gd0e6828a 4a918172a497 69d9b1fb8c61 2024-09-11T08:13:11.590 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi196 running (3m) 12s ago 3m 44.0M 3760M 17.2.7-1623-gd0e6828a 4a918172a497 41643ceb1725 2024-09-11T08:13:11.590 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi196 running (2m) 12s ago 2m 45.4M 3760M 17.2.7-1623-gd0e6828a 4a918172a497 3692287ce2d0 2024-09-11T08:13:11.590 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6 smithi196 running (2m) 12s ago 2m 42.9M 3760M 17.2.7-1623-gd0e6828a 4a918172a497 075fb35c1215 2024-09-11T08:13:11.590 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7 smithi196 running (2m) 12s ago 2m 46.2M 3760M 17.2.7-1623-gd0e6828a 4a918172a497 4ffe46806a5a 2024-09-11T08:13:11.590 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.a smithi196 *:9095 running (76s) 12s ago 76s 30.4M - 2.43.0 a07b618ecd1d 78dd43000771 2024-09-11T08:13:11.590 INFO:teuthology.orchestra.run.smithi071.stdout:rgw.foo.a smithi071 *:80 running (108s) 11s ago 108s 67.1M - 17.2.7-1623-gd0e6828a 4a918172a497 3f1ac611d9b9 2024-09-11T08:13:12.253 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:13:12 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug there is no tcmu-runner data available 2024-09-11T08:13:12.342 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-11T08:13:12.588 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:13:12.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:12 smithi071 ceph-mon[39897]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 986 B/s rd, 0 op/s; 58 B/s, 2 objects/s recovering 2024-09-11T08:13:12.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:12 smithi071 ceph-mon[39897]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:12.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:12 smithi071 ceph-mon[25834]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 986 B/s rd, 0 op/s; 58 B/s, 2 objects/s recovering 2024-09-11T08:13:12.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:12 smithi071 ceph-mon[25834]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:12.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:12 smithi196 ceph-mon[30677]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 986 B/s rd, 0 op/s; 58 B/s, 2 objects/s recovering 2024-09-11T08:13:12.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:12 smithi196 ceph-mon[30677]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:13.681 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:13 smithi196 systemd[1]: Stopping Ceph prometheus.a for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:13:13.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:13 smithi196 ceph-mon[30677]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:13.698 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:13 smithi071 ceph-mon[25834]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:13.698 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:13 smithi071 ceph-mon[39897]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:14.131 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:13:13.823Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:13:14.131 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:13:13.823Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T08:13:14.132 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:13:13.823Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T08:13:14.132 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:13:13.823Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T08:13:14.132 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:13:13.823Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T08:13:14.132 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:13:13.823Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T08:13:14.132 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:13:13.823Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T08:13:14.132 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:13:13.823Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T08:13:14.132 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:13:13.823Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T08:13:14.132 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:13:13.824Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T08:13:14.132 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:13:13.824Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T08:13:14.132 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[70358]: ts=2024-09-11T08:13:13.824Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T08:13:14.132 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:13 smithi196 podman[73675]: 2024-09-11 08:13:13.834238406 +0000 UTC m=+0.133394299 container died 78dd4300077191a963615b3d717b299efe268ab87f3ae7bb65adf944afc17d31 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:13:14.383 INFO:teuthology.orchestra.run.smithi071.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-11T08:13:14.383 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager ?:9093,9094 0/1 - 91s smithi071=a;count:1 2024-09-11T08:13:14.383 INFO:teuthology.orchestra.run.smithi071.stdout:grafana ?:3000 1/1 15s ago 87s smithi196=a;count:1 2024-09-11T08:13:14.383 INFO:teuthology.orchestra.run.smithi071.stdout:iscsi.datapool ?:5000 1/1 15s ago 102s smithi196=iscsi.a;count:1 2024-09-11T08:13:14.383 INFO:teuthology.orchestra.run.smithi071.stdout:mgr 2/2 15s ago 5m smithi071=y;smithi196=x;count:2 2024-09-11T08:13:14.384 INFO:teuthology.orchestra.run.smithi071.stdout:mon 3/3 15s ago 5m smithi071:172.21.15.71=a;smithi071:[v2:172.21.15.71:3301,v1:172.21.15.71:6790]=c;smithi196:172.21.15.196=b;count:3 2024-09-11T08:13:14.384 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter ?:9100 2/2 15s ago 95s smithi071=a;smithi196=b;count:2 2024-09-11T08:13:14.384 INFO:teuthology.orchestra.run.smithi071.stdout:osd 8 15s ago - 2024-09-11T08:13:14.384 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus ?:9095 1/1 15s ago 98s smithi196=a;count:1 2024-09-11T08:13:14.384 INFO:teuthology.orchestra.run.smithi071.stdout:rgw.foo.a ?:80 1/1 14s ago 110s smithi071=foo.a;count:1 2024-09-11T08:13:14.431 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:14 smithi196 podman[73675]: 2024-09-11 08:13:14.237989577 +0000 UTC m=+0.537145469 container cleanup 78dd4300077191a963615b3d717b299efe268ab87f3ae7bb65adf944afc17d31 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:13:14.431 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:14 smithi196 bash[73675]: ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T08:13:14.669 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:14 smithi071 ceph-mon[39897]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 297 B/s wr, 5 op/s; 53 B/s, 1 objects/s recovering 2024-09-11T08:13:14.670 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:14 smithi071 ceph-mon[25834]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 297 B/s wr, 5 op/s; 53 B/s, 1 objects/s recovering 2024-09-11T08:13:14.681 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:14 smithi196 podman[73692]: 2024-09-11 08:13:14.43072988 +0000 UTC m=+0.598358534 container remove 78dd4300077191a963615b3d717b299efe268ab87f3ae7bb65adf944afc17d31 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:13:14.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:14 smithi196 ceph-mon[30677]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 297 B/s wr, 5 op/s; 53 B/s, 1 objects/s recovering 2024-09-11T08:13:15.053 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:14 smithi196 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-11T08:13:15.054 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:14 smithi196 systemd[1]: Stopped Ceph prometheus.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:13:15.054 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:14 smithi196 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.131s CPU time. 2024-09-11T08:13:15.054 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:14 smithi196 systemd[1]: Starting Ceph prometheus.a for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:13:15.054 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:14 smithi196 podman[73800]: 2024-09-11 08:13:14.794459735 +0000 UTC m=+0.016789829 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-11T08:13:15.054 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:14 smithi196 podman[73800]: 2024-09-11 08:13:14.932551639 +0000 UTC m=+0.154881734 container create 773f05764e5aa283989e9a2e45620cb1a080a1854b9b6eec52262165ab13f3cb (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:13:15.121 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-11T08:13:15.385 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:13:15.424 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:15] ENGINE Bus STOPPING 2024-09-11T08:13:15.424 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:15] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:13:15.424 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:15] ENGINE Bus STOPPED 2024-09-11T08:13:15.431 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 podman[73800]: 2024-09-11 08:13:15.176243974 +0000 UTC m=+0.398574068 container init 773f05764e5aa283989e9a2e45620cb1a080a1854b9b6eec52262165ab13f3cb (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:13:15.432 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 podman[73800]: 2024-09-11 08:13:15.181157601 +0000 UTC m=+0.403487695 container start 773f05764e5aa283989e9a2e45620cb1a080a1854b9b6eec52262165ab13f3cb (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:13:15.432 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.205Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T08:13:15.432 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.205Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T08:13:15.432 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.205Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi196 (none))" 2024-09-11T08:13:15.432 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.205Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T08:13:15.433 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.205Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T08:13:15.433 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.207Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T08:13:15.433 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.207Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T08:13:15.433 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.208Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T08:13:15.433 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.208Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T08:13:15.434 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.210Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T08:13:15.434 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.210Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.257µs 2024-09-11T08:13:15.434 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.210Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T08:13:15.434 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.211Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-11T08:13:15.434 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.211Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-11T08:13:15.435 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.211Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=19.745µs wal_replay_duration=570.405µs wbl_replay_duration=110ns total_replay_duration=626.766µs 2024-09-11T08:13:15.435 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.212Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T08:13:15.435 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.212Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T08:13:15.435 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.212Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T08:13:15.435 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.226Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.519065ms db_storage=729ns remote_storage=1.005µs web_handler=657ns query_engine=500ns scrape=371.882µs scrape_sd=29.529µs notify=24.333µs notify_sd=7.606µs rules=13.849629ms tracing=6.268µs 2024-09-11T08:13:15.435 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.226Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T08:13:15.436 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:13:15.226Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T08:13:15.436 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 bash[73800]: 773f05764e5aa283989e9a2e45620cb1a080a1854b9b6eec52262165ab13f3cb 2024-09-11T08:13:15.436 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:13:15 smithi196 systemd[1]: Started Ceph prometheus.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:13:15.681 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[39897]: from='client.24641 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:15.681 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:15.682 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:15.682 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:13:15.682 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi071.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:13:15.682 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:15.682 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:13:15.683 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:13:15.683 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:15.683 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:13:15.683 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:13:15.683 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:15.683 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:15.684 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[25834]: from='client.24641 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:15.684 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:15.684 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:15.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:13:15.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi071.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:13:15.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:15.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:13:15.685 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:13:15.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:15.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:13:15.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:13:15.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:15.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:15.687 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:15] ENGINE Bus STARTING 2024-09-11T08:13:15.687 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:15] ENGINE Serving on http://:::9283 2024-09-11T08:13:15.688 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:15] ENGINE Bus STARTED 2024-09-11T08:13:15.688 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:15 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:15] ENGINE Bus STOPPING 2024-09-11T08:13:15.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-mon[30677]: from='client.24641 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:15.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:15.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:15.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:13:15.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi071.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:13:15.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:15.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:13:15.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:13:15.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:15.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:13:15.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:13:15.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:15.932 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:15 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:16.409 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:16] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:13:16.409 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:16] ENGINE Bus STOPPED 2024-09-11T08:13:16.409 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:16] ENGINE Bus STARTING 2024-09-11T08:13:16.409 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:16] ENGINE Serving on http://:::9283 2024-09-11T08:13:16.409 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:16] ENGINE Bus STARTED 2024-09-11T08:13:16.409 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:16] ENGINE Bus STOPPING 2024-09-11T08:13:16.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:16] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:13:16.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:16] ENGINE Bus STOPPED 2024-09-11T08:13:16.695 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:16] ENGINE Bus STARTING 2024-09-11T08:13:16.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-mon[39897]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 255 B/s wr, 4 op/s; 45 B/s, 1 objects/s recovering 2024-09-11T08:13:16.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-mon[39897]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:13:16.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-mon[39897]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi071.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:13:16.697 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-mon[39897]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:13:16.697 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-mon[39897]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:13:16.697 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-mon[39897]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:13:16.697 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-mon[39897]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:13:16.697 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:16.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-mon[25834]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 255 B/s wr, 4 op/s; 45 B/s, 1 objects/s recovering 2024-09-11T08:13:16.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-mon[25834]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:13:16.698 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-mon[25834]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi071.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:13:16.698 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-mon[25834]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:13:16.698 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-mon[25834]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:13:16.698 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-mon[25834]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:13:16.698 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-mon[25834]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:13:16.698 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:16.915 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:16 smithi196 ceph-mon[30677]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 255 B/s wr, 4 op/s; 45 B/s, 1 objects/s recovering 2024-09-11T08:13:16.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:16 smithi196 ceph-mon[30677]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:13:16.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:16 smithi196 ceph-mon[30677]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi071.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:13:16.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:16 smithi196 ceph-mon[30677]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:13:16.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:16 smithi196 ceph-mon[30677]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:13:16.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:16 smithi196 ceph-mon[30677]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:13:16.917 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:16 smithi196 ceph-mon[30677]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:13:16.917 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:16 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:17.096 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:16] ENGINE Serving on http://:::9283 2024-09-11T08:13:17.096 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:16 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:16] ENGINE Bus STARTED 2024-09-11T08:13:18.070 INFO:teuthology.orchestra.run.smithi071.stdout:HOST ADDR LABELS STATUS 2024-09-11T08:13:18.071 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071 172.21.15.71 2024-09-11T08:13:18.071 INFO:teuthology.orchestra.run.smithi071.stdout:smithi196 172.21.15.196 2024-09-11T08:13:18.071 INFO:teuthology.orchestra.run.smithi071.stdout:2 hosts in cluster 2024-09-11T08:13:18.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:18 smithi196 ceph-mon[30677]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 255 B/s wr, 4 op/s; 45 B/s, 1 objects/s recovering 2024-09-11T08:13:18.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:18 smithi071 ceph-mon[25834]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 255 B/s wr, 4 op/s; 45 B/s, 1 objects/s recovering 2024-09-11T08:13:18.695 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:13:18 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a[86704]: ts=2024-09-11T08:13:18.417Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002110354s 2024-09-11T08:13:18.696 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:18 smithi071 ceph-mon[39897]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 255 B/s wr, 4 op/s; 45 B/s, 1 objects/s recovering 2024-09-11T08:13:18.816 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-11T08:13:19.755 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:13:19.878 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:19 smithi071 ceph-mon[39897]: from='client.24647 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:19.879 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:19 smithi071 ceph-mon[25834]: from='client.24647 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:19.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:19 smithi196 ceph-mon[30677]: from='client.24647 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:20.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:20 smithi196 ceph-mon[30677]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:20.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:20 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:20.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:20 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:20.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:20 smithi071 ceph-mon[39897]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:20.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:20 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:20.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:20 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:20.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:20 smithi071 ceph-mon[25834]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:20.945 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:20 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:20.946 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:20 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:22.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:22 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:22.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:22 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:22.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:22 smithi071 ceph-mon[39897]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:22.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:22 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:22.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:22 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:22.196 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:22 smithi071 ceph-mon[25834]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:22.297 INFO:teuthology.orchestra.run.smithi071.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-11T08:13:22.298 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040172400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-11T08:13:22.298 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071 /dev/nvme1n1 ssd Linux_e546ccad77aaa03a064c 89.4G No 3m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-11T08:13:22.298 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071 /dev/nvme2n1 ssd Linux_ad37d5b30696ccfca3ce 89.4G No 3m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-11T08:13:22.298 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071 /dev/nvme3n1 ssd Linux_bd57c9dc53e965b94ed7 89.4G No 3m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-11T08:13:22.299 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071 /dev/nvme4n1 ssd Linux_af01dfb17c76d89c6286 89.4G No 3m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-11T08:13:22.299 INFO:teuthology.orchestra.run.smithi071.stdout:smithi196 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-11T08:13:22.299 INFO:teuthology.orchestra.run.smithi071.stdout:smithi196 /dev/nvme1n1 ssd Linux_d2b7d9f4cf5553bccfc7 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-11T08:13:22.299 INFO:teuthology.orchestra.run.smithi071.stdout:smithi196 /dev/nvme2n1 ssd Linux_767a095da268aa803e9d 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-11T08:13:22.299 INFO:teuthology.orchestra.run.smithi071.stdout:smithi196 /dev/nvme3n1 ssd Linux_ee8c8ccdac0431c94aaf 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-11T08:13:22.299 INFO:teuthology.orchestra.run.smithi071.stdout:smithi196 /dev/nvme4n1 ssd Linux_c718f8db9a8aac448b93 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-11T08:13:22.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:13:22 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug there is no tcmu-runner data available 2024-09-11T08:13:22.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:22 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:22.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:22 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:22.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:22 smithi196 ceph-mon[30677]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:22.987 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls --format yaml' 2024-09-11T08:13:23.407 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:23 smithi071 ceph-mon[39897]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:23.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:23 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:23.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:23 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:23.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:23 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:23.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:23 smithi071 ceph-mon[39897]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:23.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:23 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:23.408 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:23 smithi071 ceph-mon[25834]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:23.408 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:23 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:23.408 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:23 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:23.409 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:23 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:23.409 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:23 smithi071 ceph-mon[25834]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:23.409 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:23 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:23.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:23 smithi196 ceph-mon[30677]: from='client.24422 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:23.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:23 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:23.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:23 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:23.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:23 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:23.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:23 smithi196 ceph-mon[30677]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:13:23.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:23 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:13:24.181 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:24 smithi196 ceph-mon[30677]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:24.245 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:24 smithi071 ceph-mon[25834]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:24.245 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:24 smithi071 ceph-mon[39897]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:25.763 INFO:teuthology.orchestra.run.smithi071.stdout:service_type: alertmanager 2024-09-11T08:13:25.763 INFO:teuthology.orchestra.run.smithi071.stdout:service_name: alertmanager 2024-09-11T08:13:25.764 INFO:teuthology.orchestra.run.smithi071.stdout:placement: 2024-09-11T08:13:25.764 INFO:teuthology.orchestra.run.smithi071.stdout: count: 1 2024-09-11T08:13:25.764 INFO:teuthology.orchestra.run.smithi071.stdout: hosts: 2024-09-11T08:13:25.764 INFO:teuthology.orchestra.run.smithi071.stdout: - smithi071=a 2024-09-11T08:13:25.764 INFO:teuthology.orchestra.run.smithi071.stdout:status: 2024-09-11T08:13:25.764 INFO:teuthology.orchestra.run.smithi071.stdout: created: '2024-09-11T08:11:43.241118Z' 2024-09-11T08:13:25.764 INFO:teuthology.orchestra.run.smithi071.stdout: last_refresh: '2024-09-11T08:13:21.058013Z' 2024-09-11T08:13:25.764 INFO:teuthology.orchestra.run.smithi071.stdout: ports: 2024-09-11T08:13:25.764 INFO:teuthology.orchestra.run.smithi071.stdout: - 9093 2024-09-11T08:13:25.764 INFO:teuthology.orchestra.run.smithi071.stdout: - 9094 2024-09-11T08:13:25.764 INFO:teuthology.orchestra.run.smithi071.stdout: running: 1 2024-09-11T08:13:25.764 INFO:teuthology.orchestra.run.smithi071.stdout: size: 1 2024-09-11T08:13:25.765 INFO:teuthology.orchestra.run.smithi071.stdout:events: 2024-09-11T08:13:25.765 INFO:teuthology.orchestra.run.smithi071.stdout:- 2024-09-11T08:12:31.402116Z service:alertmanager [INFO] "service was created" 2024-09-11T08:13:25.765 INFO:teuthology.orchestra.run.smithi071.stdout:--- 2024-09-11T08:13:25.765 INFO:teuthology.orchestra.run.smithi071.stdout:service_type: grafana 2024-09-11T08:13:25.765 INFO:teuthology.orchestra.run.smithi071.stdout:service_name: grafana 2024-09-11T08:13:25.765 INFO:teuthology.orchestra.run.smithi071.stdout:placement: 2024-09-11T08:13:25.765 INFO:teuthology.orchestra.run.smithi071.stdout: count: 1 2024-09-11T08:13:25.765 INFO:teuthology.orchestra.run.smithi071.stdout: hosts: 2024-09-11T08:13:25.765 INFO:teuthology.orchestra.run.smithi071.stdout: - smithi196=a 2024-09-11T08:13:25.765 INFO:teuthology.orchestra.run.smithi071.stdout:spec: 2024-09-11T08:13:25.765 INFO:teuthology.orchestra.run.smithi071.stdout: anonymous_access: true 2024-09-11T08:13:25.765 INFO:teuthology.orchestra.run.smithi071.stdout:status: 2024-09-11T08:13:25.765 INFO:teuthology.orchestra.run.smithi071.stdout: created: '2024-09-11T08:11:47.229886Z' 2024-09-11T08:13:25.766 INFO:teuthology.orchestra.run.smithi071.stdout: last_refresh: '2024-09-11T08:13:19.561235Z' 2024-09-11T08:13:25.766 INFO:teuthology.orchestra.run.smithi071.stdout: ports: 2024-09-11T08:13:25.766 INFO:teuthology.orchestra.run.smithi071.stdout: - 3000 2024-09-11T08:13:25.766 INFO:teuthology.orchestra.run.smithi071.stdout: running: 1 2024-09-11T08:13:25.766 INFO:teuthology.orchestra.run.smithi071.stdout: size: 1 2024-09-11T08:13:25.766 INFO:teuthology.orchestra.run.smithi071.stdout:events: 2024-09-11T08:13:25.766 INFO:teuthology.orchestra.run.smithi071.stdout:- 2024-09-11T08:12:43.822442Z service:grafana [INFO] "service was created" 2024-09-11T08:13:25.766 INFO:teuthology.orchestra.run.smithi071.stdout:--- 2024-09-11T08:13:25.766 INFO:teuthology.orchestra.run.smithi071.stdout:service_type: iscsi 2024-09-11T08:13:25.766 INFO:teuthology.orchestra.run.smithi071.stdout:service_id: datapool 2024-09-11T08:13:25.766 INFO:teuthology.orchestra.run.smithi071.stdout:service_name: iscsi.datapool 2024-09-11T08:13:25.766 INFO:teuthology.orchestra.run.smithi071.stdout:placement: 2024-09-11T08:13:25.766 INFO:teuthology.orchestra.run.smithi071.stdout: count: 1 2024-09-11T08:13:25.766 INFO:teuthology.orchestra.run.smithi071.stdout: hosts: 2024-09-11T08:13:25.767 INFO:teuthology.orchestra.run.smithi071.stdout: - smithi196=iscsi.a 2024-09-11T08:13:25.767 INFO:teuthology.orchestra.run.smithi071.stdout:spec: 2024-09-11T08:13:25.767 INFO:teuthology.orchestra.run.smithi071.stdout: api_password: admin 2024-09-11T08:13:25.767 INFO:teuthology.orchestra.run.smithi071.stdout: api_port: 5000 2024-09-11T08:13:25.767 INFO:teuthology.orchestra.run.smithi071.stdout: api_user: admin 2024-09-11T08:13:25.767 INFO:teuthology.orchestra.run.smithi071.stdout: pool: datapool 2024-09-11T08:13:25.767 INFO:teuthology.orchestra.run.smithi071.stdout: trusted_ip_list: 172.21.15.196 2024-09-11T08:13:25.767 INFO:teuthology.orchestra.run.smithi071.stdout:status: 2024-09-11T08:13:25.767 INFO:teuthology.orchestra.run.smithi071.stdout: created: '2024-09-11T08:11:31.819797Z' 2024-09-11T08:13:25.767 INFO:teuthology.orchestra.run.smithi071.stdout: last_refresh: '2024-09-11T08:13:19.561094Z' 2024-09-11T08:13:25.767 INFO:teuthology.orchestra.run.smithi071.stdout: ports: 2024-09-11T08:13:25.767 INFO:teuthology.orchestra.run.smithi071.stdout: - 5000 2024-09-11T08:13:25.767 INFO:teuthology.orchestra.run.smithi071.stdout: running: 1 2024-09-11T08:13:25.767 INFO:teuthology.orchestra.run.smithi071.stdout: size: 1 2024-09-11T08:13:25.767 INFO:teuthology.orchestra.run.smithi071.stdout:--- 2024-09-11T08:13:25.768 INFO:teuthology.orchestra.run.smithi071.stdout:service_type: mgr 2024-09-11T08:13:25.768 INFO:teuthology.orchestra.run.smithi071.stdout:service_name: mgr 2024-09-11T08:13:25.768 INFO:teuthology.orchestra.run.smithi071.stdout:placement: 2024-09-11T08:13:25.768 INFO:teuthology.orchestra.run.smithi071.stdout: count: 2 2024-09-11T08:13:25.768 INFO:teuthology.orchestra.run.smithi071.stdout: hosts: 2024-09-11T08:13:25.768 INFO:teuthology.orchestra.run.smithi071.stdout: - smithi071=y 2024-09-11T08:13:25.768 INFO:teuthology.orchestra.run.smithi071.stdout: - smithi196=x 2024-09-11T08:13:25.768 INFO:teuthology.orchestra.run.smithi071.stdout:status: 2024-09-11T08:13:25.768 INFO:teuthology.orchestra.run.smithi071.stdout: created: '2024-09-11T08:07:56.575688Z' 2024-09-11T08:13:25.768 INFO:teuthology.orchestra.run.smithi071.stdout: last_refresh: '2024-09-11T08:13:19.560741Z' 2024-09-11T08:13:25.768 INFO:teuthology.orchestra.run.smithi071.stdout: running: 2 2024-09-11T08:13:25.768 INFO:teuthology.orchestra.run.smithi071.stdout: size: 2 2024-09-11T08:13:25.768 INFO:teuthology.orchestra.run.smithi071.stdout:--- 2024-09-11T08:13:25.769 INFO:teuthology.orchestra.run.smithi071.stdout:service_type: mon 2024-09-11T08:13:25.769 INFO:teuthology.orchestra.run.smithi071.stdout:service_name: mon 2024-09-11T08:13:25.769 INFO:teuthology.orchestra.run.smithi071.stdout:placement: 2024-09-11T08:13:25.769 INFO:teuthology.orchestra.run.smithi071.stdout: count: 3 2024-09-11T08:13:25.769 INFO:teuthology.orchestra.run.smithi071.stdout: hosts: 2024-09-11T08:13:25.769 INFO:teuthology.orchestra.run.smithi071.stdout: - smithi071:172.21.15.71=a 2024-09-11T08:13:25.769 INFO:teuthology.orchestra.run.smithi071.stdout: - smithi071:[v2:172.21.15.71:3301,v1:172.21.15.71:6790]=c 2024-09-11T08:13:25.769 INFO:teuthology.orchestra.run.smithi071.stdout: - smithi196:172.21.15.196=b 2024-09-11T08:13:25.769 INFO:teuthology.orchestra.run.smithi071.stdout:status: 2024-09-11T08:13:25.769 INFO:teuthology.orchestra.run.smithi071.stdout: created: '2024-09-11T08:07:28.608594Z' 2024-09-11T08:13:25.769 INFO:teuthology.orchestra.run.smithi071.stdout: last_refresh: '2024-09-11T08:13:19.560587Z' 2024-09-11T08:13:25.769 INFO:teuthology.orchestra.run.smithi071.stdout: running: 3 2024-09-11T08:13:25.770 INFO:teuthology.orchestra.run.smithi071.stdout: size: 3 2024-09-11T08:13:25.770 INFO:teuthology.orchestra.run.smithi071.stdout:--- 2024-09-11T08:13:25.770 INFO:teuthology.orchestra.run.smithi071.stdout:service_type: node-exporter 2024-09-11T08:13:25.770 INFO:teuthology.orchestra.run.smithi071.stdout:service_name: node-exporter 2024-09-11T08:13:25.770 INFO:teuthology.orchestra.run.smithi071.stdout:placement: 2024-09-11T08:13:25.770 INFO:teuthology.orchestra.run.smithi071.stdout: count: 2 2024-09-11T08:13:25.770 INFO:teuthology.orchestra.run.smithi071.stdout: hosts: 2024-09-11T08:13:25.770 INFO:teuthology.orchestra.run.smithi071.stdout: - smithi071=a 2024-09-11T08:13:25.770 INFO:teuthology.orchestra.run.smithi071.stdout: - smithi196=b 2024-09-11T08:13:25.770 INFO:teuthology.orchestra.run.smithi071.stdout:status: 2024-09-11T08:13:25.770 INFO:teuthology.orchestra.run.smithi071.stdout: created: '2024-09-11T08:11:39.304279Z' 2024-09-11T08:13:25.770 INFO:teuthology.orchestra.run.smithi071.stdout: last_refresh: '2024-09-11T08:13:19.561295Z' 2024-09-11T08:13:25.770 INFO:teuthology.orchestra.run.smithi071.stdout: ports: 2024-09-11T08:13:25.771 INFO:teuthology.orchestra.run.smithi071.stdout: - 9100 2024-09-11T08:13:25.771 INFO:teuthology.orchestra.run.smithi071.stdout: running: 2 2024-09-11T08:13:25.771 INFO:teuthology.orchestra.run.smithi071.stdout: size: 2 2024-09-11T08:13:25.771 INFO:teuthology.orchestra.run.smithi071.stdout:events: 2024-09-11T08:13:25.771 INFO:teuthology.orchestra.run.smithi071.stdout:- 2024-09-11T08:12:54.144084Z service:node-exporter [INFO] "service was created" 2024-09-11T08:13:25.771 INFO:teuthology.orchestra.run.smithi071.stdout:--- 2024-09-11T08:13:25.771 INFO:teuthology.orchestra.run.smithi071.stdout:service_type: osd 2024-09-11T08:13:25.771 INFO:teuthology.orchestra.run.smithi071.stdout:service_name: osd 2024-09-11T08:13:25.771 INFO:teuthology.orchestra.run.smithi071.stdout:unmanaged: true 2024-09-11T08:13:25.771 INFO:teuthology.orchestra.run.smithi071.stdout:spec: 2024-09-11T08:13:25.771 INFO:teuthology.orchestra.run.smithi071.stdout: filter_logic: AND 2024-09-11T08:13:25.771 INFO:teuthology.orchestra.run.smithi071.stdout: objectstore: bluestore 2024-09-11T08:13:25.771 INFO:teuthology.orchestra.run.smithi071.stdout:status: 2024-09-11T08:13:25.771 INFO:teuthology.orchestra.run.smithi071.stdout: container_image_id: 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T08:13:25.772 INFO:teuthology.orchestra.run.smithi071.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:13:25.772 INFO:teuthology.orchestra.run.smithi071.stdout: last_refresh: '2024-09-11T08:13:19.560817Z' 2024-09-11T08:13:25.772 INFO:teuthology.orchestra.run.smithi071.stdout: running: 8 2024-09-11T08:13:25.772 INFO:teuthology.orchestra.run.smithi071.stdout: size: 8 2024-09-11T08:13:25.772 INFO:teuthology.orchestra.run.smithi071.stdout:--- 2024-09-11T08:13:25.772 INFO:teuthology.orchestra.run.smithi071.stdout:service_type: prometheus 2024-09-11T08:13:25.772 INFO:teuthology.orchestra.run.smithi071.stdout:service_name: prometheus 2024-09-11T08:13:25.772 INFO:teuthology.orchestra.run.smithi071.stdout:placement: 2024-09-11T08:13:25.772 INFO:teuthology.orchestra.run.smithi071.stdout: count: 1 2024-09-11T08:13:25.772 INFO:teuthology.orchestra.run.smithi071.stdout: hosts: 2024-09-11T08:13:25.772 INFO:teuthology.orchestra.run.smithi071.stdout: - smithi196=a 2024-09-11T08:13:25.772 INFO:teuthology.orchestra.run.smithi071.stdout:status: 2024-09-11T08:13:25.773 INFO:teuthology.orchestra.run.smithi071.stdout: created: '2024-09-11T08:11:35.660973Z' 2024-09-11T08:13:25.773 INFO:teuthology.orchestra.run.smithi071.stdout: last_refresh: '2024-09-11T08:13:19.561158Z' 2024-09-11T08:13:25.773 INFO:teuthology.orchestra.run.smithi071.stdout: ports: 2024-09-11T08:13:25.773 INFO:teuthology.orchestra.run.smithi071.stdout: - 9095 2024-09-11T08:13:25.773 INFO:teuthology.orchestra.run.smithi071.stdout: running: 1 2024-09-11T08:13:25.773 INFO:teuthology.orchestra.run.smithi071.stdout: size: 1 2024-09-11T08:13:25.773 INFO:teuthology.orchestra.run.smithi071.stdout:events: 2024-09-11T08:13:25.773 INFO:teuthology.orchestra.run.smithi071.stdout:- 2024-09-11T08:12:54.150654Z service:prometheus [INFO] "service was created" 2024-09-11T08:13:25.773 INFO:teuthology.orchestra.run.smithi071.stdout:--- 2024-09-11T08:13:25.773 INFO:teuthology.orchestra.run.smithi071.stdout:service_type: rgw 2024-09-11T08:13:25.773 INFO:teuthology.orchestra.run.smithi071.stdout:service_id: foo.a 2024-09-11T08:13:25.773 INFO:teuthology.orchestra.run.smithi071.stdout:service_name: rgw.foo.a 2024-09-11T08:13:25.773 INFO:teuthology.orchestra.run.smithi071.stdout:placement: 2024-09-11T08:13:25.774 INFO:teuthology.orchestra.run.smithi071.stdout: count: 1 2024-09-11T08:13:25.774 INFO:teuthology.orchestra.run.smithi071.stdout: hosts: 2024-09-11T08:13:25.774 INFO:teuthology.orchestra.run.smithi071.stdout: - smithi071=foo.a 2024-09-11T08:13:25.774 INFO:teuthology.orchestra.run.smithi071.stdout:status: 2024-09-11T08:13:25.774 INFO:teuthology.orchestra.run.smithi071.stdout: created: '2024-09-11T08:11:23.758197Z' 2024-09-11T08:13:25.774 INFO:teuthology.orchestra.run.smithi071.stdout: last_refresh: '2024-09-11T08:13:21.057918Z' 2024-09-11T08:13:25.774 INFO:teuthology.orchestra.run.smithi071.stdout: ports: 2024-09-11T08:13:25.774 INFO:teuthology.orchestra.run.smithi071.stdout: - 80 2024-09-11T08:13:25.774 INFO:teuthology.orchestra.run.smithi071.stdout: running: 1 2024-09-11T08:13:25.774 INFO:teuthology.orchestra.run.smithi071.stdout: size: 1 2024-09-11T08:13:26.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:26 smithi196 ceph-mon[30677]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:26.444 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:26 smithi071 ceph-mon[39897]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:26.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:26 smithi071 ceph-mon[25834]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:26.445 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:26 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: ::ffff:172.21.15.196 - - [11/Sep/2024:08:13:26] "GET /metrics HTTP/1.1" 200 219713 "" "Prometheus/2.43.0" 2024-09-11T08:13:26.523 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 566367e6-7014-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-09-11T08:13:26.931 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:13:26 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x[32760]: ::ffff:172.21.15.196 - - [11/Sep/2024:08:13:26] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:13:27.296 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:13:27.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:27 smithi071 ceph-mon[25834]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-11T08:13:27.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:27 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:27.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:27 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:27.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:27 smithi071 ceph-mon[39897]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-11T08:13:27.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:27 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:27.695 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:27 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:27.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:27 smithi196 ceph-mon[30677]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-11T08:13:27.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:27 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:27.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:27 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:28.688 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:28 smithi071 ceph-mon[39897]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:28.688 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:28 smithi071 ceph-mon[25834]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:28.931 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:28 smithi196 ceph-mon[30677]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:29.649 INFO:teuthology.orchestra.run.smithi071.stdout:osd 8 3s ago - 2024-09-11T08:13:30.364 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:29 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:30.364 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:29 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:30.364 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:29 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:30.364 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:29 smithi071 ceph-mon[39897]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:30.364 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:29 smithi071 ceph-mon[39897]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:30.364 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:29 smithi071 ceph-mon[39897]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:30.364 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:29 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:30.364 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:29 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:30.364 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:29 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:30.364 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:29 smithi071 ceph-mon[25834]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:30.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:29 smithi071 ceph-mon[25834]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:30.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:29 smithi071 ceph-mon[25834]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:30.387 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T08:13:30.399 INFO:tasks.cephadm:Teardown begin 2024-09-11T08:13:30.399 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:13:30.427 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:13:30.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:29 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:30.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:29 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:30.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:29 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:30.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:29 smithi196 ceph-mon[30677]: from='mgr.14547 172.21.15.71:0/2948302969' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:30.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:29 smithi196 ceph-mon[30677]: from='mgr.14547 ' entity='mgr.y' 2024-09-11T08:13:30.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:29 smithi196 ceph-mon[30677]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:13:30.462 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T08:13:30.462 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T08:13:30.482 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T08:13:30.520 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T08:13:30.520 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T08:13:30.520 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-566367e6-7014-11ef-bcea-c7b262605968@mon.a 2024-09-11T08:13:30.695 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:30 smithi071 systemd[1]: Stopping Ceph mon.a for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:13:31.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:30 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-a[25811]: 2024-09-11T08:13:30.859+0000 7f5e871b4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:13:31.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:30 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-a[25811]: 2024-09-11T08:13:30.859+0000 7f5e871b4640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T08:13:31.195 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:30 smithi071 podman[89257]: 2024-09-11 08:13:30.962719298 +0000 UTC m=+0.243113553 container died e6731b5b742dd722fd8592e46902bb9992bc9df713c2cc77ea95078a242e7a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:13:31.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:31 smithi071 podman[89257]: 2024-09-11 08:13:31.400419807 +0000 UTC m=+0.680814061 container cleanup e6731b5b742dd722fd8592e46902bb9992bc9df713c2cc77ea95078a242e7a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:13:31.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:31 smithi071 bash[89257]: ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-a 2024-09-11T08:13:31.821 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T08:13:31.844 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:31 smithi071 podman[89275]: 2024-09-11 08:13:31.593854333 +0000 UTC m=+0.630086213 container remove e6731b5b742dd722fd8592e46902bb9992bc9df713c2cc77ea95078a242e7a9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T08:13:31.844 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:31 smithi071 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T08:13:31.845 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:31 smithi071 systemd[1]: Stopped Ceph mon.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:13:31.845 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 08:13:31 smithi071 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@mon.a.service: Consumed 9.566s CPU time. 2024-09-11T08:13:31.875 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:13:31.875 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T08:13:31.875 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-11T08:13:31.875 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-566367e6-7014-11ef-bcea-c7b262605968@mon.c 2024-09-11T08:13:32.195 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:31 smithi071 systemd[1]: Stopping Ceph mon.c for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:13:32.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:13:32 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug there is no tcmu-runner data available 2024-09-11T08:13:32.598 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:32 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:32] ENGINE Bus STOPPING 2024-09-11T08:13:32.598 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:32 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-c[39874]: 2024-09-11T08:13:32.215+0000 7f4dec10e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:13:32.598 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:32 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-c[39874]: 2024-09-11T08:13:32.215+0000 7f4dec10e640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T08:13:32.598 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:32 smithi071 podman[89390]: 2024-09-11 08:13:32.338894116 +0000 UTC m=+0.245220053 container died e22f17c69619cff087364d7eb77b8a5cb27047399bbe2c6a3bea9f7847d397a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-11T08:13:32.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:32 smithi071 podman[89390]: 2024-09-11 08:13:32.697022102 +0000 UTC m=+0.603348082 container cleanup e22f17c69619cff087364d7eb77b8a5cb27047399bbe2c6a3bea9f7847d397a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T08:13:32.945 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 08:13:32 smithi071 bash[89390]: ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-c 2024-09-11T08:13:32.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:32 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:32] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:13:32.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:32 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:32] ENGINE Bus STOPPED 2024-09-11T08:13:32.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:32 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:32] ENGINE Bus STARTING 2024-09-11T08:13:32.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:32 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:32] ENGINE Serving on http://:::9283 2024-09-11T08:13:32.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:32 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: [11/Sep/2024:08:13:32] ENGINE Bus STARTED 2024-09-11T08:13:33.120 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T08:13:33.155 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:13:33.155 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-11T08:13:33.155 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-11T08:13:33.156 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-566367e6-7014-11ef-bcea-c7b262605968@mon.b 2024-09-11T08:13:33.451 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:33 smithi196 systemd[1]: Stopping Ceph mon.b for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:13:33.706 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:33 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-b[30654]: 2024-09-11T08:13:33.449+0000 7f8d61e70640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:13:33.706 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:33 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-b[30654]: 2024-09-11T08:13:33.449+0000 7f8d61e70640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T08:13:34.000 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:33 smithi196 podman[75276]: 2024-09-11 08:13:33.705642029 +0000 UTC m=+0.352382527 container died 218d41aee37ff08574dbdf28bd06fd6169dc0cc98449cd5380691af70aa982ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-b, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T08:13:34.299 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:34 smithi196 podman[75276]: 2024-09-11 08:13:34.073486426 +0000 UTC m=+0.720226989 container cleanup 218d41aee37ff08574dbdf28bd06fd6169dc0cc98449cd5380691af70aa982ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:13:34.300 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 08:13:34 smithi196 bash[75276]: ceph-566367e6-7014-11ef-bcea-c7b262605968-mon-b 2024-09-11T08:13:34.444 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T08:13:34.495 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:13:34.495 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-11T08:13:34.495 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T08:13:34.495 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-566367e6-7014-11ef-bcea-c7b262605968@mgr.y 2024-09-11T08:13:34.820 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:34 smithi071 systemd[1]: Stopping Ceph mgr.y for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:13:35.160 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:34 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:13:34.818+0000 7f81bd5b9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:13:35.160 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:34 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y[26089]: 2024-09-11T08:13:34.818+0000 7f81bd5b9640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-11T08:13:35.160 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:34 smithi071 podman[89530]: 2024-09-11 08:13:34.850866778 +0000 UTC m=+0.160274066 container died da69cb3a1c3552a832d87ece080928af865908965ba867eec3c7348c89c09aa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T08:13:35.445 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:35 smithi071 podman[89530]: 2024-09-11 08:13:35.34301291 +0000 UTC m=+0.652420228 container cleanup da69cb3a1c3552a832d87ece080928af865908965ba867eec3c7348c89c09aa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-11T08:13:35.445 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 08:13:35 smithi071 bash[89530]: ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-y 2024-09-11T08:13:35.767 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T08:13:35.801 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:13:35.802 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T08:13:35.802 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T08:13:35.802 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-566367e6-7014-11ef-bcea-c7b262605968@mgr.x 2024-09-11T08:13:36.150 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:13:35 smithi196 systemd[1]: Stopping Ceph mgr.x for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:13:36.431 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:13:36 smithi196 podman[75411]: 2024-09-11 08:13:36.150371374 +0000 UTC m=+0.149868636 container died 2cc3e7f0e1d3ced938feb1cb198c0b64d20ba5d57814883be4efb0d7ee83733f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:13:36.796 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:13:36 smithi196 podman[75411]: 2024-09-11 08:13:36.510735683 +0000 UTC m=+0.510232935 container cleanup 2cc3e7f0e1d3ced938feb1cb198c0b64d20ba5d57814883be4efb0d7ee83733f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-11T08:13:36.796 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 08:13:36 smithi196 bash[75411]: ceph-566367e6-7014-11ef-bcea-c7b262605968-mgr-x 2024-09-11T08:13:36.934 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T08:13:36.970 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:13:36.970 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T08:13:36.970 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T08:13:36.971 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.0 2024-09-11T08:13:37.267 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:13:37 smithi071 systemd[1]: Stopping Ceph osd.0 for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:13:37.695 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:13:37 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-0[44270]: 2024-09-11T08:13:37.265+0000 7fb17b774640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:13:37.695 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:13:37 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-0[44270]: 2024-09-11T08:13:37.265+0000 7fb17b774640 -1 osd.0 71 *** Got signal Terminated *** 2024-09-11T08:13:37.695 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:13:37 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-0[44270]: 2024-09-11T08:13:37.265+0000 7fb17b774640 -1 osd.0 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:13:43.040 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:13:42 smithi071 podman[89658]: 2024-09-11 08:13:42.738404671 +0000 UTC m=+5.572124161 container died e3a76e1084fdae1ea46bde04c3f62e017b81ede2042e95fc81ac413f94477068 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:13:43.445 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:13:43 smithi071 podman[89658]: 2024-09-11 08:13:43.240569108 +0000 UTC m=+6.074288641 container cleanup e3a76e1084fdae1ea46bde04c3f62e017b81ede2042e95fc81ac413f94477068 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:13:43.445 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:13:43 smithi071 bash[89658]: ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-0 2024-09-11T08:13:43.908 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:13:43 smithi071 podman[89680]: 2024-09-11 08:13:43.465633238 +0000 UTC m=+0.725344255 container remove e3a76e1084fdae1ea46bde04c3f62e017b81ede2042e95fc81ac413f94477068 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:13:44.195 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:13:43 smithi071 podman[89773]: 2024-09-11 08:13:43.810891662 +0000 UTC m=+0.015661712 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:13:44.195 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:13:43 smithi071 podman[89773]: 2024-09-11 08:13:43.908252613 +0000 UTC m=+0.113022654 container create 3121e8ff3bf7434b11292f48230df5df1f567661be8ccf6e1bfab351c52b78fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T08:13:44.195 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:13:44 smithi071 podman[89773]: 2024-09-11 08:13:44.118564546 +0000 UTC m=+0.323334595 container init 3121e8ff3bf7434b11292f48230df5df1f567661be8ccf6e1bfab351c52b78fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T08:13:44.195 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:13:44 smithi071 podman[89773]: 2024-09-11 08:13:44.123465426 +0000 UTC m=+0.328235473 container start 3121e8ff3bf7434b11292f48230df5df1f567661be8ccf6e1bfab351c52b78fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T08:13:44.588 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:13:44 smithi071 podman[89773]: 2024-09-11 08:13:44.235269015 +0000 UTC m=+0.440039078 container attach 3121e8ff3bf7434b11292f48230df5df1f567661be8ccf6e1bfab351c52b78fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-11T08:13:44.588 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 08:13:44 smithi071 podman[89773]: 2024-09-11 08:13:44.29997718 +0000 UTC m=+0.504747221 container died 3121e8ff3bf7434b11292f48230df5df1f567661be8ccf6e1bfab351c52b78fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T08:13:44.985 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T08:13:45.027 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:13:45.028 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T08:13:45.028 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T08:13:45.028 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.1 2024-09-11T08:13:45.378 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:13:45 smithi071 systemd[1]: Stopping Ceph osd.1 for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:13:45.694 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:13:45 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-1[51856]: 2024-09-11T08:13:45.377+0000 7f13e3642640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:13:45.695 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:13:45 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-1[51856]: 2024-09-11T08:13:45.377+0000 7f13e3642640 -1 osd.1 71 *** Got signal Terminated *** 2024-09-11T08:13:45.695 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:13:45 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-1[51856]: 2024-09-11T08:13:45.377+0000 7f13e3642640 -1 osd.1 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:13:50.945 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:13:50 smithi071 podman[89902]: 2024-09-11 08:13:50.694497264 +0000 UTC m=+5.446549664 container died 8d8386f10e1f9cf5989ed83eacb2ac7e9bf657514e96494fe13e41008f34fd5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True) 2024-09-11T08:13:51.402 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:13:51 smithi071 podman[89902]: 2024-09-11 08:13:51.191715294 +0000 UTC m=+5.943767693 container cleanup 8d8386f10e1f9cf5989ed83eacb2ac7e9bf657514e96494fe13e41008f34fd5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-1, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2) 2024-09-11T08:13:51.402 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:13:51 smithi071 bash[89902]: ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-1 2024-09-11T08:13:51.694 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:13:51 smithi071 podman[89924]: 2024-09-11 08:13:51.401984332 +0000 UTC m=+0.702348250 container remove 8d8386f10e1f9cf5989ed83eacb2ac7e9bf657514e96494fe13e41008f34fd5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:13:52.113 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:13:51 smithi071 podman[90018]: 2024-09-11 08:13:51.702957127 +0000 UTC m=+0.015614335 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:13:52.113 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:13:51 smithi071 podman[90018]: 2024-09-11 08:13:51.81090785 +0000 UTC m=+0.123565058 container create 9ef76b7688b803a3a1a1b1df63a1e73f41336e21aab088728c247b6c5a53eb30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:13:52.114 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:13:52 smithi071 podman[90018]: 2024-09-11 08:13:52.013189297 +0000 UTC m=+0.325846507 container init 9ef76b7688b803a3a1a1b1df63a1e73f41336e21aab088728c247b6c5a53eb30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:13:52.114 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:13:52 smithi071 podman[90018]: 2024-09-11 08:13:52.018348489 +0000 UTC m=+0.331005699 container start 9ef76b7688b803a3a1a1b1df63a1e73f41336e21aab088728c247b6c5a53eb30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-1-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T08:13:52.114 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:13:52 smithi071 podman[90018]: 2024-09-11 08:13:52.11303271 +0000 UTC m=+0.425689946 container attach 9ef76b7688b803a3a1a1b1df63a1e73f41336e21aab088728c247b6c5a53eb30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T08:13:52.445 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 08:13:52 smithi071 podman[90018]: 2024-09-11 08:13:52.204866926 +0000 UTC m=+0.517524129 container died 9ef76b7688b803a3a1a1b1df63a1e73f41336e21aab088728c247b6c5a53eb30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731) 2024-09-11T08:13:52.875 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T08:13:52.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:13:52.915 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T08:13:52.915 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T08:13:52.915 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.2 2024-09-11T08:13:53.195 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:13:53 smithi071 systemd[1]: Stopping Ceph osd.2 for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:13:53.694 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:13:53 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-2[60429]: 2024-09-11T08:13:53.255+0000 7f66c6adc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:13:53.695 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:13:53 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-2[60429]: 2024-09-11T08:13:53.255+0000 7f66c6adc640 -1 osd.2 71 *** Got signal Terminated *** 2024-09-11T08:13:53.695 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:13:53 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-2[60429]: 2024-09-11T08:13:53.255+0000 7f66c6adc640 -1 osd.2 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:13:59.014 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:13:58 smithi071 podman[90145]: 2024-09-11 08:13:58.711797157 +0000 UTC m=+5.567097927 container died 88e63e05de7d3a4d98936ec270def913b256dd01d146d9919f76ad68f46552a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:13:59.379 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:13:59 smithi071 podman[90145]: 2024-09-11 08:13:59.177702239 +0000 UTC m=+6.033002991 container cleanup 88e63e05de7d3a4d98936ec270def913b256dd01d146d9919f76ad68f46552a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T08:13:59.379 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:13:59 smithi071 bash[90145]: ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-2 2024-09-11T08:13:59.694 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:13:59 smithi071 podman[90165]: 2024-09-11 08:13:59.378903272 +0000 UTC m=+0.667734477 container remove 88e63e05de7d3a4d98936ec270def913b256dd01d146d9919f76ad68f46552a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-11T08:14:00.074 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:13:59 smithi071 podman[90258]: 2024-09-11 08:13:59.692669934 +0000 UTC m=+0.015454787 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:14:00.075 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:13:59 smithi071 podman[90258]: 2024-09-11 08:13:59.821148052 +0000 UTC m=+0.143932902 container create 5d0e2e3e37f2cb2cd6a1490c29df5b927e64bf9a31e718442287841f49e00568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-11T08:14:00.445 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:14:00 smithi071 podman[90258]: 2024-09-11 08:14:00.074444746 +0000 UTC m=+0.397229595 container init 5d0e2e3e37f2cb2cd6a1490c29df5b927e64bf9a31e718442287841f49e00568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731) 2024-09-11T08:14:00.445 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:14:00 smithi071 podman[90258]: 2024-09-11 08:14:00.079654769 +0000 UTC m=+0.402439623 container start 5d0e2e3e37f2cb2cd6a1490c29df5b927e64bf9a31e718442287841f49e00568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-11T08:14:00.445 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:14:00 smithi071 podman[90258]: 2024-09-11 08:14:00.182654416 +0000 UTC m=+0.505439284 container attach 5d0e2e3e37f2cb2cd6a1490c29df5b927e64bf9a31e718442287841f49e00568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-11T08:14:00.445 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 08:14:00 smithi071 podman[90258]: 2024-09-11 08:14:00.264456289 +0000 UTC m=+0.587241139 container died 5d0e2e3e37f2cb2cd6a1490c29df5b927e64bf9a31e718442287841f49e00568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T08:14:00.846 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T08:14:00.885 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:14:00.885 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T08:14:00.885 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T08:14:00.885 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.3 2024-09-11T08:14:01.195 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:14:00 smithi071 systemd[1]: Stopping Ceph osd.3 for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:14:01.464 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:14:01 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-3[69590]: 2024-09-11T08:14:01.206+0000 7ff0ce21f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:14:01.464 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:14:01 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-3[69590]: 2024-09-11T08:14:01.206+0000 7ff0ce21f640 -1 osd.3 71 *** Got signal Terminated *** 2024-09-11T08:14:01.464 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:14:01 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-3[69590]: 2024-09-11T08:14:01.206+0000 7ff0ce21f640 -1 osd.3 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:14:03.431 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:03 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:03.152+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:04.431 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:04 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:04.145+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:04.944 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:14:04 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-3[69590]: 2024-09-11T08:14:04.615+0000 7ff0c7831640 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.414487+0000 front 2024-09-11T08:13:39.414655+0000 (oldest deadline 2024-09-11T08:14:04.114702+0000) 2024-09-11T08:14:05.452 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:05 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4[36098]: 2024-09-11T08:14:05.434+0000 7fee5b3df640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.682986+0000 front 2024-09-11T08:13:41.682916+0000 (oldest deadline 2024-09-11T08:14:05.182925+0000) 2024-09-11T08:14:05.452 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:05 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:05.187+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:05.945 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:14:05 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-3[69590]: 2024-09-11T08:14:05.621+0000 7ff0c7831640 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.414487+0000 front 2024-09-11T08:13:39.414655+0000 (oldest deadline 2024-09-11T08:14:04.114702+0000) 2024-09-11T08:14:06.583 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:06 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:06.218+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:06.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:06 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4[36098]: 2024-09-11T08:14:06.455+0000 7fee5b3df640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.682986+0000 front 2024-09-11T08:13:41.682916+0000 (oldest deadline 2024-09-11T08:14:05.182925+0000) 2024-09-11T08:14:06.887 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:14:06 smithi071 podman[90395]: 2024-09-11 08:14:06.597024537 +0000 UTC m=+5.491722276 container died 69d9b1fb8c6173dd8203f39d4c77929a1d288c211c6cb5120818f35351b1415e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T08:14:06.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:06 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:06.669+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.285528+0000 front 2024-09-11T08:13:41.285653+0000 (oldest deadline 2024-09-11T08:14:06.585611+0000) 2024-09-11T08:14:06.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:06 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:06.582+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:07.197 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:14:07 smithi071 podman[90395]: 2024-09-11 08:14:07.06268004 +0000 UTC m=+5.957377775 container cleanup 69d9b1fb8c6173dd8203f39d4c77929a1d288c211c6cb5120818f35351b1415e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-11T08:14:07.197 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:14:07 smithi071 bash[90395]: ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-3 2024-09-11T08:14:07.487 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:07 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:07.236+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:07.614 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:14:07 smithi071 podman[90416]: 2024-09-11 08:14:07.297060065 +0000 UTC m=+0.697549903 container remove 69d9b1fb8c6173dd8203f39d4c77929a1d288c211c6cb5120818f35351b1415e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:14:07.925 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:14:07 smithi071 podman[90511]: 2024-09-11 08:14:07.614372255 +0000 UTC m=+0.108297956 container create 49c28851b44bebde42c6bffee31f68ee5b3ba53520fc32c09c9b25fdaf69bd81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-11T08:14:07.925 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:14:07 smithi071 podman[90511]: 2024-09-11 08:14:07.521855675 +0000 UTC m=+0.015781388 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:14:07.925 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:14:07 smithi071 podman[90511]: 2024-09-11 08:14:07.841735183 +0000 UTC m=+0.335660889 container init 49c28851b44bebde42c6bffee31f68ee5b3ba53520fc32c09c9b25fdaf69bd81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T08:14:07.925 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:14:07 smithi071 podman[90511]: 2024-09-11 08:14:07.846231051 +0000 UTC m=+0.340156752 container start 49c28851b44bebde42c6bffee31f68ee5b3ba53520fc32c09c9b25fdaf69bd81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T08:14:07.926 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:14:07 smithi071 podman[90511]: 2024-09-11 08:14:07.92469821 +0000 UTC m=+0.418623924 container attach 49c28851b44bebde42c6bffee31f68ee5b3ba53520fc32c09c9b25fdaf69bd81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T08:14:07.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:07 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4[36098]: 2024-09-11T08:14:07.486+0000 7fee5b3df640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.682986+0000 front 2024-09-11T08:13:41.682916+0000 (oldest deadline 2024-09-11T08:14:05.182925+0000) 2024-09-11T08:14:07.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:07 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:07.625+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.285528+0000 front 2024-09-11T08:13:41.285653+0000 (oldest deadline 2024-09-11T08:14:06.585611+0000) 2024-09-11T08:14:07.932 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:07 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:07.550+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:08.195 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 08:14:08 smithi071 podman[90511]: 2024-09-11 08:14:08.02448249 +0000 UTC m=+0.518408191 container died 49c28851b44bebde42c6bffee31f68ee5b3ba53520fc32c09c9b25fdaf69bd81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-11T08:14:08.595 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:08 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:08.263+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:08.595 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:08 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4[36098]: 2024-09-11T08:14:08.472+0000 7fee5b3df640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.682986+0000 front 2024-09-11T08:13:41.682916+0000 (oldest deadline 2024-09-11T08:14:05.182925+0000) 2024-09-11T08:14:08.761 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T08:14:08.800 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:14:08.800 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T08:14:08.800 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T08:14:08.801 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.4 2024-09-11T08:14:08.858 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:08 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:08.636+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.285528+0000 front 2024-09-11T08:13:41.285653+0000 (oldest deadline 2024-09-11T08:14:06.585611+0000) 2024-09-11T08:14:08.858 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:08 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:08.594+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:09.181 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:08 smithi196 systemd[1]: Stopping Ceph osd.4 for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:14:09.181 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:09 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4[36098]: 2024-09-11T08:14:09.105+0000 7fee61dcd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:14:09.181 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:09 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4[36098]: 2024-09-11T08:14:09.105+0000 7fee61dcd640 -1 osd.4 71 *** Got signal Terminated *** 2024-09-11T08:14:09.181 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:09 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4[36098]: 2024-09-11T08:14:09.105+0000 7fee61dcd640 -1 osd.4 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:14:09.596 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:09 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:09.266+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:09.596 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:09 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4[36098]: 2024-09-11T08:14:09.426+0000 7fee5b3df640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.682986+0000 front 2024-09-11T08:13:41.682916+0000 (oldest deadline 2024-09-11T08:14:05.182925+0000) 2024-09-11T08:14:09.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:09 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:09.595+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.285528+0000 front 2024-09-11T08:13:41.285653+0000 (oldest deadline 2024-09-11T08:14:06.585611+0000) 2024-09-11T08:14:09.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:09 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:09.617+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:10.579 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:10 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:10.228+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:10.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:10 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4[36098]: 2024-09-11T08:14:10.392+0000 7fee5b3df640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.682986+0000 front 2024-09-11T08:13:41.682916+0000 (oldest deadline 2024-09-11T08:14:05.182925+0000) 2024-09-11T08:14:10.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:10 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:10.578+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.285528+0000 front 2024-09-11T08:13:41.285653+0000 (oldest deadline 2024-09-11T08:14:06.585611+0000) 2024-09-11T08:14:10.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:10 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:10.610+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:11.571 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:11 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:11.260+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:11.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:11 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4[36098]: 2024-09-11T08:14:11.417+0000 7fee5b3df640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.682986+0000 front 2024-09-11T08:13:41.682916+0000 (oldest deadline 2024-09-11T08:14:05.182925+0000) 2024-09-11T08:14:11.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:11 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:11.650+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:11.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:11 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:11.570+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.285528+0000 front 2024-09-11T08:13:41.285653+0000 (oldest deadline 2024-09-11T08:14:06.585611+0000) 2024-09-11T08:14:11.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:11 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:11.570+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.586207+0000 front 2024-09-11T08:13:46.586143+0000 (oldest deadline 2024-09-11T08:14:11.285985+0000) 2024-09-11T08:14:12.589 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:12 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:12.230+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:12.589 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:12 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:12.230+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:12.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:12 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4[36098]: 2024-09-11T08:14:12.430+0000 7fee5b3df640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.682986+0000 front 2024-09-11T08:13:41.682916+0000 (oldest deadline 2024-09-11T08:14:05.182925+0000) 2024-09-11T08:14:12.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:12 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4[36098]: 2024-09-11T08:14:12.430+0000 7fee5b3df640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:47.483791+0000 front 2024-09-11T08:13:47.483719+0000 (oldest deadline 2024-09-11T08:14:11.583633+0000) 2024-09-11T08:14:12.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:12 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:12.649+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:12.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:12 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:12.649+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:12.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:12 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:12.588+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.285528+0000 front 2024-09-11T08:13:41.285653+0000 (oldest deadline 2024-09-11T08:14:06.585611+0000) 2024-09-11T08:14:12.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:12 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:12.588+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.586207+0000 front 2024-09-11T08:13:46.586143+0000 (oldest deadline 2024-09-11T08:14:11.285985+0000) 2024-09-11T08:14:13.608 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:13.273+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:13.609 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:13.273+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:13.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4[36098]: 2024-09-11T08:14:13.420+0000 7fee5b3df640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.682986+0000 front 2024-09-11T08:13:41.682916+0000 (oldest deadline 2024-09-11T08:14:05.182925+0000) 2024-09-11T08:14:13.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4[36098]: 2024-09-11T08:14:13.420+0000 7fee5b3df640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:47.483791+0000 front 2024-09-11T08:13:47.483719+0000 (oldest deadline 2024-09-11T08:14:11.583633+0000) 2024-09-11T08:14:13.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:13.643+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:13.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:13.643+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:13.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:13.607+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.285528+0000 front 2024-09-11T08:13:41.285653+0000 (oldest deadline 2024-09-11T08:14:06.585611+0000) 2024-09-11T08:14:13.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:13 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:13.607+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.586207+0000 front 2024-09-11T08:13:46.586143+0000 (oldest deadline 2024-09-11T08:14:11.285985+0000) 2024-09-11T08:14:14.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:14 smithi196 podman[75556]: 2024-09-11 08:14:14.451600356 +0000 UTC m=+5.446424929 container died 41643ceb172579c47c29bfee4b8c26d6f7a18988d448340f8fe628b898c43cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T08:14:14.589 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:14 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:14.306+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:14.589 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:14 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:14.306+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:14.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:14 smithi196 podman[75556]: 2024-09-11 08:14:14.786236736 +0000 UTC m=+5.781061313 container cleanup 41643ceb172579c47c29bfee4b8c26d6f7a18988d448340f8fe628b898c43cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-11T08:14:14.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:14 smithi196 bash[75556]: ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4 2024-09-11T08:14:14.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:14 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:14.587+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.285528+0000 front 2024-09-11T08:13:41.285653+0000 (oldest deadline 2024-09-11T08:14:06.585611+0000) 2024-09-11T08:14:14.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:14 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:14.587+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.586207+0000 front 2024-09-11T08:13:46.586143+0000 (oldest deadline 2024-09-11T08:14:11.285985+0000) 2024-09-11T08:14:14.932 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:14 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:14.612+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:14.932 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:14 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:14.612+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:15.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:14 smithi196 podman[75576]: 2024-09-11 08:14:14.979108273 +0000 UTC m=+0.523780282 container remove 41643ceb172579c47c29bfee4b8c26d6f7a18988d448340f8fe628b898c43cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T08:14:15.526 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:15.261+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:15.526 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:15.261+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:15.527 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:15 smithi196 podman[75668]: 2024-09-11 08:14:15.291566259 +0000 UTC m=+0.015731319 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:14:15.527 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:15 smithi196 podman[75668]: 2024-09-11 08:14:15.41319075 +0000 UTC m=+0.137355798 container create faa49f973cebc7b7cc5a260d4ff26f00ec1edab5231e5371fff0c3c0f5f06b06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:14:15.821 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:15.548+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.285528+0000 front 2024-09-11T08:13:41.285653+0000 (oldest deadline 2024-09-11T08:14:06.585611+0000) 2024-09-11T08:14:15.822 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:15.548+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.586207+0000 front 2024-09-11T08:13:46.586143+0000 (oldest deadline 2024-09-11T08:14:11.285985+0000) 2024-09-11T08:14:15.822 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:15.583+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:15.822 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:15 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:15.583+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:15.822 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:15 smithi196 podman[75668]: 2024-09-11 08:14:15.63260433 +0000 UTC m=+0.356769376 container init faa49f973cebc7b7cc5a260d4ff26f00ec1edab5231e5371fff0c3c0f5f06b06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T08:14:15.822 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:15 smithi196 podman[75668]: 2024-09-11 08:14:15.637512672 +0000 UTC m=+0.361677735 container start faa49f973cebc7b7cc5a260d4ff26f00ec1edab5231e5371fff0c3c0f5f06b06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T08:14:15.823 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:15 smithi196 podman[75668]: 2024-09-11 08:14:15.715739768 +0000 UTC m=+0.439904824 container attach faa49f973cebc7b7cc5a260d4ff26f00ec1edab5231e5371fff0c3c0f5f06b06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T08:14:16.118 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 08:14:15 smithi196 podman[75668]: 2024-09-11 08:14:15.821840449 +0000 UTC m=+0.546005507 container died faa49f973cebc7b7cc5a260d4ff26f00ec1edab5231e5371fff0c3c0f5f06b06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-11T08:14:16.405 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:16 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:16.288+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:16.406 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:16 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:16.288+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:16.418 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T08:14:16.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:14:16.458 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T08:14:16.458 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T08:14:16.458 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.5 2024-09-11T08:14:16.681 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:16 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:16.617+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:16.681 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:16 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:16.617+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:16.681 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:16 smithi196 systemd[1]: Stopping Ceph osd.5 for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:14:16.681 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:16 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:16.586+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.285528+0000 front 2024-09-11T08:13:41.285653+0000 (oldest deadline 2024-09-11T08:14:06.585611+0000) 2024-09-11T08:14:16.682 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:16 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:16.586+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.586207+0000 front 2024-09-11T08:13:46.586143+0000 (oldest deadline 2024-09-11T08:14:11.285985+0000) 2024-09-11T08:14:17.181 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:16 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:16.782+0000 7fe562b14640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:14:17.182 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:16 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:16.782+0000 7fe562b14640 -1 osd.5 71 *** Got signal Terminated *** 2024-09-11T08:14:17.182 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:16 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:16.782+0000 7fe562b14640 -1 osd.5 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:14:17.579 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:17 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:17.325+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:17.579 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:17 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:17.325+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:17.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:17 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:17.621+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:17.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:17 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:17.621+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:17.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:17 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:17.588+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.285528+0000 front 2024-09-11T08:13:41.285653+0000 (oldest deadline 2024-09-11T08:14:06.585611+0000) 2024-09-11T08:14:17.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:17 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:17.588+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.586207+0000 front 2024-09-11T08:13:46.586143+0000 (oldest deadline 2024-09-11T08:14:11.285985+0000) 2024-09-11T08:14:18.651 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:18 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:18.597+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.285528+0000 front 2024-09-11T08:13:41.285653+0000 (oldest deadline 2024-09-11T08:14:06.585611+0000) 2024-09-11T08:14:18.651 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:18 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:18.597+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.586207+0000 front 2024-09-11T08:13:46.586143+0000 (oldest deadline 2024-09-11T08:14:11.285985+0000) 2024-09-11T08:14:18.651 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:18 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:18.352+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:18.652 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:18 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:18.352+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:18.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:18 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:18.650+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:18.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:18 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:18.650+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:19.681 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:19 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:19.379+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:19.681 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:19 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:19.379+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:19.681 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:19 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:19.628+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:19.682 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:19 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:19.628+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:19.682 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:19 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:19.602+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.285528+0000 front 2024-09-11T08:13:41.285653+0000 (oldest deadline 2024-09-11T08:14:06.585611+0000) 2024-09-11T08:14:19.682 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:19 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:19.602+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.586207+0000 front 2024-09-11T08:13:46.586143+0000 (oldest deadline 2024-09-11T08:14:11.285985+0000) 2024-09-11T08:14:20.681 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:20 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:20.413+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:20.681 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:20 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:20.413+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:20.681 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:20 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:20.607+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:20.681 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:20 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:20.607+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:20.682 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:20 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:20.553+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.285528+0000 front 2024-09-11T08:13:41.285653+0000 (oldest deadline 2024-09-11T08:14:06.585611+0000) 2024-09-11T08:14:20.682 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:20 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:20.553+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.586207+0000 front 2024-09-11T08:13:46.586143+0000 (oldest deadline 2024-09-11T08:14:11.285985+0000) 2024-09-11T08:14:20.682 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:20 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:20.553+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:54.087272+0000 front 2024-09-11T08:13:54.087359+0000 (oldest deadline 2024-09-11T08:14:19.987180+0000) 2024-09-11T08:14:21.681 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:21 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:21.397+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:21.681 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:21 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:21.397+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:21.681 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:21 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:21.610+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:21.681 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:21 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:21.610+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:21.681 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:21 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:21.553+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:41.285528+0000 front 2024-09-11T08:13:41.285653+0000 (oldest deadline 2024-09-11T08:14:06.585611+0000) 2024-09-11T08:14:21.682 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:21 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:21.553+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.586207+0000 front 2024-09-11T08:13:46.586143+0000 (oldest deadline 2024-09-11T08:14:11.285985+0000) 2024-09-11T08:14:21.682 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:21 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5[43864]: 2024-09-11T08:14:21.553+0000 7fe55c126640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:54.087272+0000 front 2024-09-11T08:13:54.087359+0000 (oldest deadline 2024-09-11T08:14:19.987180+0000) 2024-09-11T08:14:22.581 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:22 smithi196 podman[75795]: 2024-09-11 08:14:22.307102478 +0000 UTC m=+5.617824145 container died 3692287ce2d005b609a56502019e89fd04aaac6277b26aa29e3e55064021aa89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:14:22.581 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:22 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:22.393+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:22.581 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:22 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:22.393+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:22.581 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:22 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:22.393+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:56.195241+0000 front 2024-09-11T08:13:56.195253+0000 (oldest deadline 2024-09-11T08:14:22.095178+0000) 2024-09-11T08:14:22.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:22 smithi196 podman[75795]: 2024-09-11 08:14:22.696837671 +0000 UTC m=+6.007559338 container cleanup 3692287ce2d005b609a56502019e89fd04aaac6277b26aa29e3e55064021aa89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:14:22.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:22 smithi196 bash[75795]: ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5 2024-09-11T08:14:22.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:22 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:22.609+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:22.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:22 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:22.609+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:23.261 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:22 smithi196 podman[75827]: 2024-09-11 08:14:22.931302105 +0000 UTC m=+0.617713570 container remove 3692287ce2d005b609a56502019e89fd04aaac6277b26aa29e3e55064021aa89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:14:23.576 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:23 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:23.415+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:23.576 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:23 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:23.415+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:23.576 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:23 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:23.415+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:56.195241+0000 front 2024-09-11T08:13:56.195253+0000 (oldest deadline 2024-09-11T08:14:22.095178+0000) 2024-09-11T08:14:23.577 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:23 smithi196 podman[75919]: 2024-09-11 08:14:23.163771251 +0000 UTC m=+0.015503468 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:14:23.577 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:23 smithi196 podman[75919]: 2024-09-11 08:14:23.265543117 +0000 UTC m=+0.117275376 container create e201508e8db36201acb34466443d0b3c4baf1dea387dc1d91509d50717c5bea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T08:14:23.577 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:23 smithi196 podman[75919]: 2024-09-11 08:14:23.493009398 +0000 UTC m=+0.344741617 container init e201508e8db36201acb34466443d0b3c4baf1dea387dc1d91509d50717c5bea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T08:14:23.577 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:23 smithi196 podman[75919]: 2024-09-11 08:14:23.498021186 +0000 UTC m=+0.349753411 container start e201508e8db36201acb34466443d0b3c4baf1dea387dc1d91509d50717c5bea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T08:14:23.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:23 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:23.575+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:23.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:23 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:23.575+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:23.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:23 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:23.575+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:57.652354+0000 front 2024-09-11T08:13:57.652287+0000 (oldest deadline 2024-09-11T08:14:23.552294+0000) 2024-09-11T08:14:23.931 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:23 smithi196 podman[75919]: 2024-09-11 08:14:23.584311752 +0000 UTC m=+0.436043978 container attach e201508e8db36201acb34466443d0b3c4baf1dea387dc1d91509d50717c5bea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:14:23.932 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 08:14:23 smithi196 podman[75919]: 2024-09-11 08:14:23.677460452 +0000 UTC m=+0.529192670 container died e201508e8db36201acb34466443d0b3c4baf1dea387dc1d91509d50717c5bea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-11T08:14:24.319 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T08:14:24.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:14:24.360 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T08:14:24.360 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T08:14:24.360 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.6 2024-09-11T08:14:24.610 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:24 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:24.429+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:24.611 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:24 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:24.429+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:24.611 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:24 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:24.429+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:56.195241+0000 front 2024-09-11T08:13:56.195253+0000 (oldest deadline 2024-09-11T08:14:22.095178+0000) 2024-09-11T08:14:24.611 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:24 smithi196 systemd[1]: Stopping Ceph osd.6 for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:14:24.931 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:24 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:24.718+0000 7efca43f1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:14:24.931 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:24 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:24.718+0000 7efca43f1640 -1 osd.6 71 *** Got signal Terminated *** 2024-09-11T08:14:24.931 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:24 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:24.718+0000 7efca43f1640 -1 osd.6 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:14:24.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:24 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:24.610+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:24.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:24 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:24.610+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:24.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:24 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:24.610+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:57.652354+0000 front 2024-09-11T08:13:57.652287+0000 (oldest deadline 2024-09-11T08:14:23.552294+0000) 2024-09-11T08:14:25.681 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:25 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:25.383+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:25.681 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:25 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:25.383+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:25.681 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:25 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:25.383+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:56.195241+0000 front 2024-09-11T08:13:56.195253+0000 (oldest deadline 2024-09-11T08:14:22.095178+0000) 2024-09-11T08:14:25.682 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:25 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:25.615+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:25.682 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:25 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:25.615+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:25.682 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:25 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:25.615+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:57.652354+0000 front 2024-09-11T08:13:57.652287+0000 (oldest deadline 2024-09-11T08:14:23.552294+0000) 2024-09-11T08:14:26.681 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:26 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:26.402+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:26.681 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:26 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:26.402+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:26.681 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:26 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:26.402+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:56.195241+0000 front 2024-09-11T08:13:56.195253+0000 (oldest deadline 2024-09-11T08:14:22.095178+0000) 2024-09-11T08:14:26.681 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:26 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:26.571+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:26.681 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:26 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:26.571+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:26.681 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:26 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:26.571+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:57.652354+0000 front 2024-09-11T08:13:57.652287+0000 (oldest deadline 2024-09-11T08:14:23.552294+0000) 2024-09-11T08:14:27.652 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:27 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:27.590+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:27.652 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:27 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:27.590+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:27.652 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:27 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:27.590+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:57.652354+0000 front 2024-09-11T08:13:57.652287+0000 (oldest deadline 2024-09-11T08:14:23.552294+0000) 2024-09-11T08:14:27.652 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:27 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:27.397+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:27.652 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:27 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:27.397+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:27.652 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:27 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:27.397+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:56.195241+0000 front 2024-09-11T08:13:56.195253+0000 (oldest deadline 2024-09-11T08:14:22.095178+0000) 2024-09-11T08:14:27.652 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:27 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:27.397+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6830 osd.3 since back 2024-09-11T08:14:02.095705+0000 front 2024-09-11T08:14:02.095726+0000 (oldest deadline 2024-09-11T08:14:26.795539+0000) 2024-09-11T08:14:28.681 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:28 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:28.420+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:28.681 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:28 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:28.420+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:28.681 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:28 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:28.420+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:56.195241+0000 front 2024-09-11T08:13:56.195253+0000 (oldest deadline 2024-09-11T08:14:22.095178+0000) 2024-09-11T08:14:28.681 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:28 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:28.420+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6830 osd.3 since back 2024-09-11T08:14:02.095705+0000 front 2024-09-11T08:14:02.095726+0000 (oldest deadline 2024-09-11T08:14:26.795539+0000) 2024-09-11T08:14:28.681 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:28 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:28.615+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:28.682 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:28 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:28.615+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:28.682 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:28 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:28.615+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:57.652354+0000 front 2024-09-11T08:13:57.652287+0000 (oldest deadline 2024-09-11T08:14:23.552294+0000) 2024-09-11T08:14:28.682 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:28 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:28.615+0000 7fea21323640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14508.0:140 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-11T08:14:29.656 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:29 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:29.369+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:39.892802+0000 front 2024-09-11T08:13:39.892705+0000 (oldest deadline 2024-09-11T08:14:02.792782+0000) 2024-09-11T08:14:29.656 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:29 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:29.369+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:45.594437+0000 front 2024-09-11T08:13:45.594305+0000 (oldest deadline 2024-09-11T08:14:11.494342+0000) 2024-09-11T08:14:29.656 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:29 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:29.369+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:56.195241+0000 front 2024-09-11T08:13:56.195253+0000 (oldest deadline 2024-09-11T08:14:22.095178+0000) 2024-09-11T08:14:29.656 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:29 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6[52359]: 2024-09-11T08:14:29.369+0000 7efc9fa07640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.71:6830 osd.3 since back 2024-09-11T08:14:02.095705+0000 front 2024-09-11T08:14:02.095726+0000 (oldest deadline 2024-09-11T08:14:26.795539+0000) 2024-09-11T08:14:29.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:29 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:29.654+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:29.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:29 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:29.654+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:29.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:29 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:29.654+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:57.652354+0000 front 2024-09-11T08:13:57.652287+0000 (oldest deadline 2024-09-11T08:14:23.552294+0000) 2024-09-11T08:14:29.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:29 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:29.654+0000 7fea21323640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14508.0:140 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-11T08:14:30.533 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:30 smithi196 podman[76046]: 2024-09-11 08:14:30.228563056 +0000 UTC m=+5.651660433 container died 075fb35c1215740c016443be64a76847b968d1d5e264b2ff4be4c2a6c3c1d700 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:14:30.875 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:30 smithi196 podman[76046]: 2024-09-11 08:14:30.657204566 +0000 UTC m=+6.080301938 container cleanup 075fb35c1215740c016443be64a76847b968d1d5e264b2ff4be4c2a6c3c1d700 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T08:14:30.875 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:30 smithi196 bash[76046]: ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6 2024-09-11T08:14:30.875 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:30 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:30.659+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:30.875 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:30 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:30.659+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:30.875 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:30 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:30.659+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:57.652354+0000 front 2024-09-11T08:13:57.652287+0000 (oldest deadline 2024-09-11T08:14:23.552294+0000) 2024-09-11T08:14:30.876 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:30 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:30.659+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6830 osd.3 since back 2024-09-11T08:14:04.653158+0000 front 2024-09-11T08:14:04.653146+0000 (oldest deadline 2024-09-11T08:14:29.952994+0000) 2024-09-11T08:14:30.876 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:30 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:30.660+0000 7fea21323640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14508.0:140 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-11T08:14:31.181 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:30 smithi196 podman[76066]: 2024-09-11 08:14:30.874854556 +0000 UTC m=+0.635777108 container remove 075fb35c1215740c016443be64a76847b968d1d5e264b2ff4be4c2a6c3c1d700 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T08:14:31.628 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:31 smithi196 podman[76162]: 2024-09-11 08:14:31.22537167 +0000 UTC m=+0.016107487 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:14:31.628 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:31 smithi196 podman[76162]: 2024-09-11 08:14:31.334107067 +0000 UTC m=+0.124842913 container create 69bf1be3344158a4e840b95b58e9a1fe54cb8ba8a7b7e1fac2fb5193d469bd43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-11T08:14:31.628 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:31 smithi196 podman[76162]: 2024-09-11 08:14:31.544687038 +0000 UTC m=+0.335422853 container init 69bf1be3344158a4e840b95b58e9a1fe54cb8ba8a7b7e1fac2fb5193d469bd43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-11T08:14:31.628 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:31 smithi196 podman[76162]: 2024-09-11 08:14:31.549460719 +0000 UTC m=+0.340196528 container start 69bf1be3344158a4e840b95b58e9a1fe54cb8ba8a7b7e1fac2fb5193d469bd43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:14:31.628 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:31 smithi196 podman[76162]: 2024-09-11 08:14:31.627936517 +0000 UTC m=+0.418672330 container attach 69bf1be3344158a4e840b95b58e9a1fe54cb8ba8a7b7e1fac2fb5193d469bd43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T08:14:31.931 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 08:14:31 smithi196 podman[76162]: 2024-09-11 08:14:31.729249768 +0000 UTC m=+0.519985573 container died 69bf1be3344158a4e840b95b58e9a1fe54cb8ba8a7b7e1fac2fb5193d469bd43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:14:31.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:31 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:31.632+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:31.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:31 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:31.632+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:31.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:31 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:31.632+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:57.652354+0000 front 2024-09-11T08:13:57.652287+0000 (oldest deadline 2024-09-11T08:14:23.552294+0000) 2024-09-11T08:14:31.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:31 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:31.632+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6830 osd.3 since back 2024-09-11T08:14:04.653158+0000 front 2024-09-11T08:14:04.653146+0000 (oldest deadline 2024-09-11T08:14:29.952994+0000) 2024-09-11T08:14:31.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:31 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:31.632+0000 7fea21323640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14508.0:140 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-11T08:14:32.284 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T08:14:32.322 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:14:32.323 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T08:14:32.323 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T08:14:32.323 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.7 2024-09-11T08:14:32.628 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:32 smithi196 systemd[1]: Stopping Ceph osd.7 for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:14:32.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:32 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:32.626+0000 7fea23508640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:14:32.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:32 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:32.626+0000 7fea23508640 -1 osd.7 71 *** Got signal Terminated *** 2024-09-11T08:14:32.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:32 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:32.626+0000 7fea23508640 -1 osd.7 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:14:32.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:32 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:32.633+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:32.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:32 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:32.633+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:32.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:32 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:32.633+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:57.652354+0000 front 2024-09-11T08:13:57.652287+0000 (oldest deadline 2024-09-11T08:14:23.552294+0000) 2024-09-11T08:14:32.932 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:32 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:32.633+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6830 osd.3 since back 2024-09-11T08:14:04.653158+0000 front 2024-09-11T08:14:04.653146+0000 (oldest deadline 2024-09-11T08:14:29.952994+0000) 2024-09-11T08:14:32.932 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:32 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:32.633+0000 7fea21323640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14508.0:140 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-11T08:14:33.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:33 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:33.659+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:33.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:33 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:33.659+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:33.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:33 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:33.659+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:57.652354+0000 front 2024-09-11T08:13:57.652287+0000 (oldest deadline 2024-09-11T08:14:23.552294+0000) 2024-09-11T08:14:33.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:33 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:33.659+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6830 osd.3 since back 2024-09-11T08:14:04.653158+0000 front 2024-09-11T08:14:04.653146+0000 (oldest deadline 2024-09-11T08:14:29.952994+0000) 2024-09-11T08:14:33.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:33 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:33.659+0000 7fea21323640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14508.0:140 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-11T08:14:34.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:34 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:34.611+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:34.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:34 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:34.611+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:34.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:34 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:34.611+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:57.652354+0000 front 2024-09-11T08:13:57.652287+0000 (oldest deadline 2024-09-11T08:14:23.552294+0000) 2024-09-11T08:14:34.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:34 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:34.611+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6830 osd.3 since back 2024-09-11T08:14:04.653158+0000 front 2024-09-11T08:14:04.653146+0000 (oldest deadline 2024-09-11T08:14:29.952994+0000) 2024-09-11T08:14:34.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:34 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:34.611+0000 7fea21323640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14508.0:140 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-11T08:14:35.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:35 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:35.571+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:35.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:35 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:35.571+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:35.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:35 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:35.571+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:57.652354+0000 front 2024-09-11T08:13:57.652287+0000 (oldest deadline 2024-09-11T08:14:23.552294+0000) 2024-09-11T08:14:35.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:35 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:35.571+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6830 osd.3 since back 2024-09-11T08:14:04.653158+0000 front 2024-09-11T08:14:04.653146+0000 (oldest deadline 2024-09-11T08:14:29.952994+0000) 2024-09-11T08:14:35.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:35 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:35.572+0000 7fea21323640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14508.0:140 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-11T08:14:36.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:36 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:36.538+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:36.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:36 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:36.538+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:36.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:36 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:36.538+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:57.652354+0000 front 2024-09-11T08:13:57.652287+0000 (oldest deadline 2024-09-11T08:14:23.552294+0000) 2024-09-11T08:14:36.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:36 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:36.538+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6830 osd.3 since back 2024-09-11T08:14:04.653158+0000 front 2024-09-11T08:14:04.653146+0000 (oldest deadline 2024-09-11T08:14:29.952994+0000) 2024-09-11T08:14:36.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:36 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:36.539+0000 7fea21323640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14508.0:140 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-11T08:14:37.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:37.512+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-11T08:13:40.551353+0000 front 2024-09-11T08:13:40.551189+0000 (oldest deadline 2024-09-11T08:14:06.451081+0000) 2024-09-11T08:14:37.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:37.512+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-09-11T08:13:46.451616+0000 front 2024-09-11T08:13:46.451721+0000 (oldest deadline 2024-09-11T08:14:12.351495+0000) 2024-09-11T08:14:37.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:37.512+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-09-11T08:13:57.652354+0000 front 2024-09-11T08:13:57.652287+0000 (oldest deadline 2024-09-11T08:14:23.552294+0000) 2024-09-11T08:14:37.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:37.512+0000 7fea21323640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.71:6830 osd.3 since back 2024-09-11T08:14:04.653158+0000 front 2024-09-11T08:14:04.653146+0000 (oldest deadline 2024-09-11T08:14:29.952994+0000) 2024-09-11T08:14:37.931 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:37 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7[61047]: 2024-09-11T08:14:37.512+0000 7fea21323640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14508.0:140 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-11T08:14:38.260 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:37 smithi196 podman[76289]: 2024-09-11 08:14:37.989142666 +0000 UTC m=+5.441461900 container died 4ffe46806a5a3e526758f9afc4c8f2d12a384915d86ab83028bb5088d3112bc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-09-11T08:14:38.681 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:38 smithi196 podman[76289]: 2024-09-11 08:14:38.450457358 +0000 UTC m=+5.902776626 container cleanup 4ffe46806a5a3e526758f9afc4c8f2d12a384915d86ab83028bb5088d3112bc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-11T08:14:38.681 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:38 smithi196 bash[76289]: ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7 2024-09-11T08:14:39.008 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:38 smithi196 podman[76311]: 2024-09-11 08:14:38.701509284 +0000 UTC m=+0.702684785 container remove 4ffe46806a5a3e526758f9afc4c8f2d12a384915d86ab83028bb5088d3112bc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:14:39.279 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:39 smithi196 podman[76403]: 2024-09-11 08:14:38.910307998 +0000 UTC m=+0.015612259 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:14:39.279 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:39 smithi196 podman[76403]: 2024-09-11 08:14:39.019056626 +0000 UTC m=+0.124360874 container create ecc1554557f9e26956f298d100beeed593fd0bc14ae164cea3796454c6c6668e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T08:14:39.280 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:39 smithi196 podman[76403]: 2024-09-11 08:14:39.204416152 +0000 UTC m=+0.309720404 container init ecc1554557f9e26956f298d100beeed593fd0bc14ae164cea3796454c6c6668e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:14:39.280 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:39 smithi196 podman[76403]: 2024-09-11 08:14:39.209341256 +0000 UTC m=+0.314645505 container start ecc1554557f9e26956f298d100beeed593fd0bc14ae164cea3796454c6c6668e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:14:39.280 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:39 smithi196 podman[76403]: 2024-09-11 08:14:39.27941891 +0000 UTC m=+0.384723167 container attach ecc1554557f9e26956f298d100beeed593fd0bc14ae164cea3796454c6c6668e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0) 2024-09-11T08:14:39.665 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 08:14:39 smithi196 podman[76403]: 2024-09-11 08:14:39.38715207 +0000 UTC m=+0.492456326 container died ecc1554557f9e26956f298d100beeed593fd0bc14ae164cea3796454c6c6668e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-11T08:14:40.043 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T08:14:40.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:14:40.082 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T08:14:40.083 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-11T08:14:40.083 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-566367e6-7014-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-11T08:14:40.386 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:14:40 smithi071 systemd[1]: Stopping Ceph rgw.foo.a for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:14:40.694 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:14:40 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-rgw-foo-a[77019]: 2024-09-11T08:14:40.384+0000 7f35377fe640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:14:40.695 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:14:40 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-rgw-foo-a[77019]: 2024-09-11T08:14:40.384+0000 7f353ffc17c0 -1 shutting down 2024-09-11T08:14:50.695 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:14:50 smithi071 bash[90644]: time="2024-09-11T08:14:50Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-566367e6-7014-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-11T08:14:50.695 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:14:50 smithi071 podman[90644]: 2024-09-11 08:14:50.404739397 +0000 UTC m=+10.128476181 container died 3f1ac611d9b91c2002dbcc8138686a44db29328b7f84b676e097661590991df2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-rgw-foo-a, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T08:14:51.054 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:14:50 smithi071 podman[90644]: 2024-09-11 08:14:50.904771183 +0000 UTC m=+10.628507976 container cleanup 3f1ac611d9b91c2002dbcc8138686a44db29328b7f84b676e097661590991df2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:14:51.055 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:14:50 smithi071 bash[90644]: ceph-566367e6-7014-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-11T08:14:51.288 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-11T08:14:51.311 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:14:51 smithi071 podman[90681]: 2024-09-11 08:14:51.15709101 +0000 UTC m=+0.752605306 container remove 3f1ac611d9b91c2002dbcc8138686a44db29328b7f84b676e097661590991df2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T08:14:51.311 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:14:51 smithi071 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-09-11T08:14:51.311 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:14:51 smithi071 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@rgw.foo.a.service: Failed with result 'exit-code'. 2024-09-11T08:14:51.311 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:14:51 smithi071 systemd[1]: Stopped Ceph rgw.foo.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:14:51.312 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 08:14:51 smithi071 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@rgw.foo.a.service: Consumed 1.502s CPU time. 2024-09-11T08:14:51.325 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:14:51.325 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-11T08:14:51.326 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-11T08:14:51.326 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-566367e6-7014-11ef-bcea-c7b262605968@prometheus.a 2024-09-11T08:14:51.619 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:14:51 smithi196 systemd[1]: Stopping Ceph prometheus.a for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:14:51.902 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:14:51 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:14:51.619Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:14:51.902 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:14:51 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:14:51.619Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T08:14:51.902 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:14:51 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:14:51.619Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T08:14:51.902 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:14:51 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:14:51.619Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T08:14:51.902 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:14:51 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:14:51.619Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T08:14:51.903 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:14:51 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:14:51.619Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T08:14:51.903 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:14:51 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:14:51.619Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T08:14:51.903 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:14:51 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:14:51.619Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T08:14:51.903 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:14:51 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:14:51.619Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T08:14:51.903 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:14:51 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:14:51.620Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T08:14:51.903 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:14:51 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:14:51.620Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T08:14:51.903 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:14:51 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a[73816]: ts=2024-09-11T08:14:51.620Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T08:14:51.903 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:14:51 smithi196 podman[76531]: 2024-09-11 08:14:51.630096421 +0000 UTC m=+0.104551233 container died 773f05764e5aa283989e9a2e45620cb1a080a1854b9b6eec52262165ab13f3cb (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:14:51.945 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:14:51 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a[86704]: ts=2024-09-11T08:14:51.626Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-09-11T08:14:51.945 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:14:51 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a[86704]: ts=2024-09-11T08:14:51.626Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-09-11T08:14:52.181 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:14:52 smithi196 podman[76531]: 2024-09-11 08:14:52.101035826 +0000 UTC m=+0.575490635 container cleanup 773f05764e5aa283989e9a2e45620cb1a080a1854b9b6eec52262165ab13f3cb (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:14:52.181 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 08:14:52 smithi196 bash[76531]: ceph-566367e6-7014-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T08:14:52.480 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-566367e6-7014-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-11T08:14:52.518 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:14:52.518 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-11T08:14:52.518 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 566367e6-7014-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T08:14:54.445 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:14:54 smithi071 systemd[1]: Stopping Ceph alertmanager.a for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:14:54.795 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:14:54 smithi071 ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a[86704]: ts=2024-09-11T08:14:54.496Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:14:54.795 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:14:54 smithi071 podman[91120]: 2024-09-11 08:14:54.517346347 +0000 UTC m=+0.127767167 container died c42e8261ece503a62f3df687fcb3709bdeead04e94bb1fe7f23255f026d89e87 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:14:55.153 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:14:54 smithi071 podman[91120]: 2024-09-11 08:14:54.90249281 +0000 UTC m=+0.512913629 container cleanup c42e8261ece503a62f3df687fcb3709bdeead04e94bb1fe7f23255f026d89e87 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:14:55.153 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:14:54 smithi071 bash[91120]: ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T08:14:55.445 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:14:55 smithi071 podman[91139]: 2024-09-11 08:14:55.153210524 +0000 UTC m=+0.636636656 container remove c42e8261ece503a62f3df687fcb3709bdeead04e94bb1fe7f23255f026d89e87 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:14:55.445 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:14:55 smithi071 podman[91139]: 2024-09-11 08:14:55.237252758 +0000 UTC m=+0.720678887 volume remove 8a0c177b444b04609e8ba923dcb79997bc56fc9c336df31491394e9e94282581 2024-09-11T08:14:55.445 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:14:55 smithi071 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T08:14:55.445 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 08:14:55 smithi071 systemd[1]: Stopped Ceph alertmanager.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:14:55.827 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:14:55 smithi071 systemd[1]: Stopping Ceph node-exporter.a for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:14:56.181 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:14:55 smithi071 podman[91252]: 2024-09-11 08:14:55.827126617 +0000 UTC m=+0.134320023 container died 4837cdf81df568b22dabb6d6344081facaa8596ff4ff61dc3751cc12516442df (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:14:56.444 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:14:56 smithi071 podman[91252]: 2024-09-11 08:14:56.272632393 +0000 UTC m=+0.579825787 container cleanup 4837cdf81df568b22dabb6d6344081facaa8596ff4ff61dc3751cc12516442df (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:14:56.445 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:14:56 smithi071 bash[91252]: ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a 2024-09-11T08:14:56.841 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:14:56 smithi071 podman[91269]: 2024-09-11 08:14:56.540029942 +0000 UTC m=+0.716724281 container remove 4837cdf81df568b22dabb6d6344081facaa8596ff4ff61dc3751cc12516442df (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:14:56.841 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:14:56 smithi071 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:14:56.841 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:14:56 smithi071 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-11T08:14:56.842 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:14:56 smithi071 systemd[1]: Stopped Ceph node-exporter.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:14:56.842 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 08:14:56 smithi071 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 1.973s CPU time. 2024-09-11T08:14:57.344 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 566367e6-7014-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T08:14:58.931 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:14:58 smithi196 systemd[1]: Stopping Ceph iscsi.iscsi.a for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:14:59.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:14:59 smithi196 ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a[68531]: debug Shutdown received 2024-09-11T08:15:09.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:15:09 smithi196 bash[76928]: time="2024-09-11T08:15:09Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-11T08:15:09.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:15:09 smithi196 podman[76928]: 2024-09-11 08:15:09.059932128 +0000 UTC m=+10.121838490 container died da81cc55fa16f4f0a5babbacd884c2af3e664af7b305b0e108998040693549e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-11T08:15:09.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:15:09 smithi196 podman[76928]: 2024-09-11 08:15:09.478669512 +0000 UTC m=+10.540575865 container cleanup da81cc55fa16f4f0a5babbacd884c2af3e664af7b305b0e108998040693549e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-11T08:15:09.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:15:09 smithi196 bash[76928]: ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-11T08:15:10.146 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:15:09 smithi196 podman[76950]: 2024-09-11 08:15:09.713130013 +0000 UTC m=+0.653067662 container remove da81cc55fa16f4f0a5babbacd884c2af3e664af7b305b0e108998040693549e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:15:10.146 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:15:09 smithi196 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-11T08:15:10.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:15:10 smithi196 podman[77000]: 2024-09-11 08:15:10.145516685 +0000 UTC m=+0.200897104 container died 6633cb4cfcf363ca313aad30c9fb66f6a96ea3ecc589472ff2a5887abbec938c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T08:15:10.793 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:15:10 smithi196 podman[77000]: 2024-09-11 08:15:10.490734155 +0000 UTC m=+0.546114616 container cleanup 6633cb4cfcf363ca313aad30c9fb66f6a96ea3ecc589472ff2a5887abbec938c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T08:15:10.793 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:15:10 smithi196 bash[77000]: ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-11T08:15:10.793 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:15:10 smithi196 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-11T08:15:10.793 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:15:10 smithi196 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 68390 (conmon) remains running after unit stopped. 2024-09-11T08:15:10.793 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:15:10 smithi196 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 77017 (podman) remains running after unit stopped. 2024-09-11T08:15:10.793 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:15:10 smithi196 systemd[1]: Stopped Ceph iscsi.iscsi.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:15:10.793 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:15:10 smithi196 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 2.363s CPU time. 2024-09-11T08:15:10.793 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 08:15:10 smithi196 podman[77017]: 2024-09-11 08:15:10.792151416 +0000 UTC m=+0.650331246 container remove 6633cb4cfcf363ca313aad30c9fb66f6a96ea3ecc589472ff2a5887abbec938c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:15:11.181 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:15:10 smithi196 systemd[1]: Stopping Ceph grafana.a for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:15:11.448 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:15:11 smithi196 podman[77142]: 2024-09-11 08:15:11.182570273 +0000 UTC m=+0.116820358 container died b20068dfb13a99b8377e0f18493035f8c7fa2c604cb9db11e3955cb4940865da (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a, architecture=x86_64, build-date=2022-04-29T11:48:11.055897, name=ubi8, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, distribution-scope=public, vendor=Red Hat, Inc., summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, io.openshift.expose-services=, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.tags=base rhel8, maintainer=Paul Cuzner , release=239.1651231664, io.buildah.version=1.32.0, vcs-type=git, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, version=8.5) 2024-09-11T08:15:11.779 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:15:11 smithi196 podman[77142]: 2024-09-11 08:15:11.594646344 +0000 UTC m=+0.528896413 container cleanup b20068dfb13a99b8377e0f18493035f8c7fa2c604cb9db11e3955cb4940865da (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a, version=8.5, description=Ceph Grafana Container, vendor=Red Hat, Inc., summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner , io.buildah.version=1.32.0, architecture=x86_64, io.openshift.expose-services=, name=ubi8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, build-date=2022-04-29T11:48:11.055897, com.redhat.component=ubi8-container, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.tags=base rhel8, release=239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56) 2024-09-11T08:15:11.780 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:15:11 smithi196 bash[77142]: ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a 2024-09-11T08:15:12.038 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:15:11 smithi196 podman[77160]: 2024-09-11 08:15:11.779016146 +0000 UTC m=+0.598667909 container remove b20068dfb13a99b8377e0f18493035f8c7fa2c604cb9db11e3955cb4940865da (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-grafana-a, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-type=git, architecture=x86_64, distribution-scope=public, io.openshift.expose-services=, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.component=ubi8-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.buildah.version=1.32.0, vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , name=ubi8, version=8.5, description=Ceph Grafana Container, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.tags=base rhel8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.) 2024-09-11T08:15:12.038 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:15:11 smithi196 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:15:12.038 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:15:12 smithi196 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-11T08:15:12.038 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:15:12 smithi196 systemd[1]: Stopped Ceph grafana.a for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:15:12.038 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 08:15:12 smithi196 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@grafana.a.service: Consumed 3.991s CPU time. 2024-09-11T08:15:12.431 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:15:12 smithi196 systemd[1]: Stopping Ceph node-exporter.b for 566367e6-7014-11ef-bcea-c7b262605968... 2024-09-11T08:15:12.725 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:15:12 smithi196 podman[77275]: 2024-09-11 08:15:12.474719774 +0000 UTC m=+0.123985923 container died 214d90a9f27e6e0c8dbf765676b5da39631661328972f08d9bac2677f29c9325 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:15:13.074 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:15:12 smithi196 podman[77275]: 2024-09-11 08:15:12.898618608 +0000 UTC m=+0.547884746 container cleanup 214d90a9f27e6e0c8dbf765676b5da39631661328972f08d9bac2677f29c9325 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:15:13.075 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:15:12 smithi196 bash[77275]: ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b 2024-09-11T08:15:13.330 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:15:13 smithi196 podman[77292]: 2024-09-11 08:15:13.074709808 +0000 UTC m=+0.601981327 container remove 214d90a9f27e6e0c8dbf765676b5da39631661328972f08d9bac2677f29c9325 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-566367e6-7014-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:15:13.330 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:15:13 smithi196 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:15:13.331 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:15:13 smithi196 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-11T08:15:13.331 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:15:13 smithi196 systemd[1]: Stopped Ceph node-exporter.b for 566367e6-7014-11ef-bcea-c7b262605968. 2024-09-11T08:15:13.331 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 08:15:13 smithi196 systemd[1]: ceph-566367e6-7014-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 1.937s CPU time. 2024-09-11T08:15:13.989 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:15:14.022 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:15:14.049 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T08:15:14.052 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901259/remote/smithi071/crash 2024-09-11T08:15:14.053 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/crash -- . 2024-09-11T08:15:14.112 INFO:teuthology.orchestra.run.smithi071.stderr:tar: /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T08:15:14.112 INFO:teuthology.orchestra.run.smithi071.stderr:tar: Error is not recoverable: exiting now 2024-09-11T08:15:14.114 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901259/remote/smithi196/crash 2024-09-11T08:15:14.115 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/crash -- . 2024-09-11T08:15:14.151 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/566367e6-7014-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T08:15:14.151 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-09-11T08:15:14.152 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T08:15:14.152 DEBUG:teuthology.orchestra.run.smithi071:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v MON_DOWN | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | head -n 1 2024-09-11T08:15:14.180 INFO:teuthology.orchestra.run.smithi071.stderr:grep: /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-11T08:15:14.181 INFO:tasks.cephadm:Compressing logs... 2024-09-11T08:15:14.181 DEBUG:teuthology.orchestra.run.smithi071:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T08:15:14.224 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T08:15:14.251 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-11T08:15:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:: No such file or directory 2024-09-11T08:15:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T08:15:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T08:15:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/tcmu-runner.log: /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T08:15:14.252 INFO:teuthology.orchestra.run.smithi196.stderr: 63.7% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-11T08:15:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T08:15:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T08:15:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T08:15:14.252 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T08:15:14.252 INFO:teuthology.orchestra.run.smithi071.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T08:15:14.252 INFO:teuthology.orchestra.run.smithi071.stderr:: No such file or directory 2024-09-11T08:15:14.253 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T08:15:14.253 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T08:15:14.253 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T08:15:14.253 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T08:15:14.253 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T08:15:14.253 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T08:15:14.253 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-11T08:15:14.253 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T08:15:14.254 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: 58.5% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-11T08:15:14.254 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T08:15:14.254 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T08:15:14.254 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T08:15:14.257 INFO:teuthology.orchestra.run.smithi196.stderr: 90.6% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T08:15:14.266 INFO:teuthology.orchestra.run.smithi196.stderr: 90.6% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T08:15:14.268 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T08:15:14.269 INFO:teuthology.orchestra.run.smithi071.stderr: 90.6% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T08:15:14.383 INFO:teuthology.orchestra.run.smithi071.stderr: 93.0% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T08:15:14.385 INFO:teuthology.orchestra.run.smithi196.stderr: 92.1% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T08:15:14.387 INFO:teuthology.orchestra.run.smithi071.stderr: 92.2% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T08:15:14.391 INFO:teuthology.orchestra.run.smithi071.stderr: 88.6% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T08:15:14.403 INFO:teuthology.orchestra.run.smithi196.stderr: 93.1% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T08:15:14.416 INFO:teuthology.orchestra.run.smithi196.stderr: 93.3% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T08:15:14.439 INFO:teuthology.orchestra.run.smithi196.stderr: 93.1% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T08:15:14.440 INFO:teuthology.orchestra.run.smithi196.stderr: 93.5% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T08:15:14.442 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-09-11T08:15:14.442 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m0.204s 2024-09-11T08:15:14.443 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m0.841s 2024-09-11T08:15:14.443 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.045s 2024-09-11T08:15:14.472 INFO:teuthology.orchestra.run.smithi071.stderr: 93.4% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T08:15:14.499 INFO:teuthology.orchestra.run.smithi071.stderr: 93.5% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T08:15:14.522 INFO:teuthology.orchestra.run.smithi071.stderr: 93.4% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T08:15:14.580 INFO:teuthology.orchestra.run.smithi071.stderr: 91.2% -- replaced with /var/log/ceph/566367e6-7014-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T08:15:14.582 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-11T08:15:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:real 0m0.344s 2024-09-11T08:15:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:user 0m1.472s 2024-09-11T08:15:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:sys 0m0.058s 2024-09-11T08:15:14.582 INFO:tasks.cephadm:Archiving logs... 2024-09-11T08:15:14.583 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901259/remote/smithi071/log 2024-09-11T08:15:14.584 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T08:15:14.843 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901259/remote/smithi196/log 2024-09-11T08:15:14.844 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T08:15:14.992 INFO:tasks.cephadm:Removing cluster... 2024-09-11T08:15:14.992 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 566367e6-7014-11ef-bcea-c7b262605968 --force 2024-09-11T08:15:15.328 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 566367e6-7014-11ef-bcea-c7b262605968 --force 2024-09-11T08:15:15.654 INFO:tasks.cephadm:Removing cephadm ... 2024-09-11T08:15:15.654 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T08:15:15.669 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T08:15:15.684 INFO:tasks.cephadm:Teardown complete 2024-09-11T08:15:15.685 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-09-11T08:15:15.695 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi071:/dev/vg_nvme/lv_1... 2024-09-11T08:15:15.695 DEBUG:teuthology.orchestra.run.smithi071:> sudo nvme disconnect -n lv_1 2024-09-11T08:15:15.860 INFO:teuthology.orchestra.run.smithi071.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-11T08:15:15.861 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi071:/dev/vg_nvme/lv_2... 2024-09-11T08:15:15.862 DEBUG:teuthology.orchestra.run.smithi071:> sudo nvme disconnect -n lv_2 2024-09-11T08:15:16.033 INFO:teuthology.orchestra.run.smithi071.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-11T08:15:16.035 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi071:/dev/vg_nvme/lv_3... 2024-09-11T08:15:16.035 DEBUG:teuthology.orchestra.run.smithi071:> sudo nvme disconnect -n lv_3 2024-09-11T08:15:16.175 INFO:teuthology.orchestra.run.smithi071.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-11T08:15:16.177 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi071:/dev/vg_nvme/lv_4... 2024-09-11T08:15:16.177 DEBUG:teuthology.orchestra.run.smithi071:> sudo nvme disconnect -n lv_4 2024-09-11T08:15:16.314 INFO:teuthology.orchestra.run.smithi071.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-11T08:15:16.316 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T08:15:16.316 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/scratch_devs 2024-09-11T08:15:16.341 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi196:/dev/vg_nvme/lv_1... 2024-09-11T08:15:16.342 DEBUG:teuthology.orchestra.run.smithi196:> sudo nvme disconnect -n lv_1 2024-09-11T08:15:16.493 INFO:teuthology.orchestra.run.smithi196.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-11T08:15:16.494 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi196:/dev/vg_nvme/lv_2... 2024-09-11T08:15:16.495 DEBUG:teuthology.orchestra.run.smithi196:> sudo nvme disconnect -n lv_2 2024-09-11T08:15:16.647 INFO:teuthology.orchestra.run.smithi196.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-11T08:15:16.649 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi196:/dev/vg_nvme/lv_3... 2024-09-11T08:15:16.650 DEBUG:teuthology.orchestra.run.smithi196:> sudo nvme disconnect -n lv_3 2024-09-11T08:15:16.824 INFO:teuthology.orchestra.run.smithi196.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-11T08:15:16.825 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi196:/dev/vg_nvme/lv_4... 2024-09-11T08:15:16.825 DEBUG:teuthology.orchestra.run.smithi196:> sudo nvme disconnect -n lv_4 2024-09-11T08:15:16.976 INFO:teuthology.orchestra.run.smithi196.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-11T08:15:16.978 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T08:15:16.978 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/scratch_devs 2024-09-11T08:15:17.002 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T08:15:17.011 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T08:15:17.011 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:15:17.013 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:15:17.026 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:15:17.030 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:15:17.030 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-09-11T08:15:17.030 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 61 -1273us[-1273us] +/- 73ms 2024-09-11T08:15:17.031 INFO:teuthology.orchestra.run.smithi071.stdout:^* hv02.front.sepia.ceph.com 3 6 377 64 +816us[ +810us] +/- 65ms 2024-09-11T08:15:17.031 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 +351us[ +351us] +/- 64ms 2024-09-11T08:15:17.031 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:15:17.057 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:15:17.061 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:15:17.061 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-09-11T08:15:17.061 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 62 -1274us[-1274us] +/- 73ms 2024-09-11T08:15:17.061 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv02.front.sepia.ceph.com 3 6 377 62 +795us[ +774us] +/- 65ms 2024-09-11T08:15:17.061 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +339us[ +339us] +/- 64ms 2024-09-11T08:15:17.061 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:15:17.062 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T08:15:17.387 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T08:15:17.388 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T08:15:17.743 DEBUG:teuthology.orchestra.run.smithi071:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T08:15:17.825 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T08:15:17.897 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:15:17.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:15:17.928 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:15:17.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:15:17.956 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T08:15:17.965 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T08:15:18.014 INFO:teuthology.task.internal:Duration was 941.037523 seconds 2024-09-11T08:15:18.014 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T08:15:18.499 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T08:15:18.499 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T08:15:18.502 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T08:15:18.542 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:15:18.545 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:15:18.984 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T08:15:18.984 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-09-11T08:15:18.984 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T08:15:19.007 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-09-11T08:15:19.007 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T08:15:19.032 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T08:15:19.032 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T08:15:19.050 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T08:15:19.101 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T08:15:19.101 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T08:15:19.119 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T08:15:19.395 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T08:15:19.406 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T08:15:19.406 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T08:15:19.431 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T08:15:19.455 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T08:15:19.521 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:15:19.523 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:15:19.544 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-09-11T08:15:19.546 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-09-11T08:15:19.570 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:15:19.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:15:19.617 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:15:19.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:15:19.632 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T08:15:19.649 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T08:15:19.650 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901259/remote/smithi071 2024-09-11T08:15:19.650 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T08:15:19.695 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901259/remote/smithi196 2024-09-11T08:15:19.695 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T08:15:19.731 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T08:15:19.731 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T08:15:19.734 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T08:15:19.780 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T08:15:19.859 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T08:15:19.859 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T08:15:20.139 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T08:15:20.139 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T08:15:20.142 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T08:15:20.154 INFO:teuthology.orchestra.run.smithi071.stdout: 266054 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 08:15 /home/ubuntu/cephtest 2024-09-11T08:15:20.156 INFO:teuthology.orchestra.run.smithi196.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 08:15 /home/ubuntu/cephtest 2024-09-11T08:15:20.157 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T08:15:20.201 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T08:15:20.447 INFO:teuthology.run:Summary data: description: orch:cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} duration: 941.0375225543976 owner: scheduled_gabrioux@teuthology success: true 2024-09-11T08:15:20.447 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T08:15:20.536 INFO:teuthology.run:pass