2024-09-11T08:22:33.659 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T08:22:33.673 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T08:22:33.762 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901292 branch: wip-guits-quincy-2024-09-10-2008-quincy description: orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} email: gabrioux@ibm.com first_in_suite: false job_id: '7901292' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-10-2008-quincy ceph: conf: global: mon election default strategy: 1 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) mon_bind_msgr2: false sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 extra_packages: - cephadm selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-quincy-2024-09-10-2008-quincy sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 2282 sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-quincy-2024-09-10-2008-quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 targets: smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEO0qtIuKBF4hyIQTHQPxKFatsVqbEIoyKdcNTjWraQtLOmx3nliUNuvt5Ps006y57xupS0ZP5YhHD7RNQv6gyo= smithi189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBo4xQQ5Iu7UwE2W7gwPMLNM4pTz4XuTnDG8Vzf/9GNi6yewKK+TY0GpS2X2gklsUfxJeurqAKXYQj3lePF98iw= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - "set -ex\nfor f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x\n\ do\n echo \"rotating key for $f\"\n K=$(ceph auth get-key $f)\n NK=\"\ $K\"\n ceph orch daemon rotate-key $f\n while [ \"$K\" == \"$NK\" ]; do\n\ \ sleep 5\n NK=$(ceph auth get-key $f)\n done\ndone\n" teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_06:53:43 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T08:22:33.763 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa; will attempt to use it 2024-09-11T08:22:33.763 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa/tasks 2024-09-11T08:22:33.763 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T08:22:33.765 INFO:teuthology.task.internal:Checking packages... 2024-09-11T08:22:33.784 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd0e6828a2016d48cf25ad84064e50742bb1c39b9' 2024-09-11T08:22:33.784 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T08:22:33.785 INFO:teuthology.packaging:ref: None 2024-09-11T08:22:33.785 INFO:teuthology.packaging:tag: None 2024-09-11T08:22:33.785 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T08:22:33.785 INFO:teuthology.packaging:sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:22:33.785 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T08:22:33.921 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1623.gd0e6828a 2024-09-11T08:22:33.923 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T08:22:33.930 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T08:22:33.930 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T08:22:33.942 INFO:teuthology.task.internal:Saving configuration 2024-09-11T08:22:33.953 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T08:22:33.980 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T08:22:34.005 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901292', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 08:17:23.821700', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEO0qtIuKBF4hyIQTHQPxKFatsVqbEIoyKdcNTjWraQtLOmx3nliUNuvt5Ps006y57xupS0ZP5YhHD7RNQv6gyo='} 2024-09-11T08:22:34.026 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901292', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 08:17:23.818879', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBo4xQQ5Iu7UwE2W7gwPMLNM4pTz4XuTnDG8Vzf/9GNi6yewKK+TY0GpS2X2gklsUfxJeurqAKXYQj3lePF98iw='} 2024-09-11T08:22:34.026 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T08:22:34.032 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-11T08:22:34.033 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-11T08:22:34.033 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T08:22:34.106 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa23e057ac0>, signals=[15]) 2024-09-11T08:22:34.107 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T08:22:34.113 INFO:teuthology.task.internal:Opening connections... 2024-09-11T08:22:34.113 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2024-09-11T08:22:34.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:22:34.192 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2024-09-11T08:22:34.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:22:34.267 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T08:22:34.275 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2024-09-11T08:22:34.293 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2024-09-11T08:22:34.293 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2024-09-11T08:22:34.351 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2024-09-11T08:22:34.352 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="9" 2024-09-11T08:22:34.352 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2024-09-11T08:22:34.352 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2024-09-11T08:22:34.352 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="9" 2024-09-11T08:22:34.352 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el9" 2024-09-11T08:22:34.352 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T08:22:34.352 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2024-09-11T08:22:34.352 INFO:teuthology.orchestra.run.smithi131.stdout:LOGO="fedora-logo-icon" 2024-09-11T08:22:34.352 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T08:22:34.352 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2024-09-11T08:22:34.353 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T08:22:34.353 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T08:22:34.353 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T08:22:34.353 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2024-09-11T08:22:34.375 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2024-09-11T08:22:34.393 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2024-09-11T08:22:34.394 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2024-09-11T08:22:34.452 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="CentOS Stream" 2024-09-11T08:22:34.452 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="9" 2024-09-11T08:22:34.452 INFO:teuthology.orchestra.run.smithi189.stdout:ID="centos" 2024-09-11T08:22:34.452 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="rhel fedora" 2024-09-11T08:22:34.452 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="9" 2024-09-11T08:22:34.452 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el9" 2024-09-11T08:22:34.453 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T08:22:34.453 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31" 2024-09-11T08:22:34.453 INFO:teuthology.orchestra.run.smithi189.stdout:LOGO="fedora-logo-icon" 2024-09-11T08:22:34.453 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T08:22:34.453 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://centos.org/" 2024-09-11T08:22:34.453 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T08:22:34.453 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T08:22:34.453 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T08:22:34.453 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2024-09-11T08:22:34.475 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T08:22:34.484 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T08:22:34.609 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T08:22:34.610 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2024-09-11T08:22:34.612 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2024-09-11T08:22:34.630 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T08:22:34.697 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T08:22:34.697 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2024-09-11T08:22:34.700 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2024-09-11T08:22:34.810 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T08:22:34.862 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T08:22:34.876 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T08:22:34.876 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-11T08:22:34.877 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T08:22:34.877 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T08:22:34.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:22:34.895 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-09-11T08:22:34.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:22:34.896 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-09-11T08:22:34.952 INFO:teuthology.orchestra.run.smithi131.stdout:5.14.0-503.el9.x86_64 2024-09-11T08:22:34.952 INFO:teuthology.task.kernel:Running kernel on smithi131: 5.14.0-503.el9.x86_64 2024-09-11T08:22:34.953 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum install -y kernel 2024-09-11T08:22:34.954 INFO:teuthology.orchestra.run.smithi189.stdout:5.14.0-503.el9.x86_64 2024-09-11T08:22:34.954 INFO:teuthology.task.kernel:Running kernel on smithi189: 5.14.0-503.el9.x86_64 2024-09-11T08:22:34.954 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum install -y kernel 2024-09-11T08:22:37.437 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-11T08:22:38.581 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.2 MB 00:01 2024-09-11T08:22:40.957 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 16 MB/s | 20 MB 00:01 2024-09-11T08:22:46.278 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-11T08:22:47.505 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.5 MB 00:02 2024-09-11T08:22:49.316 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-11T08:22:50.170 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-11T08:22:50.779 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 9.4 MB/s | 6.5 MB 00:00 2024-09-11T08:22:52.463 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-11T08:22:53.217 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-11T08:22:55.693 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-11T08:22:57.699 INFO:teuthology.orchestra.run.smithi189.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T08:22:57.699 INFO:teuthology.orchestra.run.smithi189.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T08:22:57.794 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:22:57.799 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:22:57.799 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-09-11T08:22:57.799 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:22:57.800 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:22:57.800 INFO:teuthology.orchestra.run.smithi189.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T08:22:57.800 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-11T08:22:57.800 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T08:22:57.800 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T08:22:57.800 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T08:22:57.800 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:22:57.800 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:22:57.800 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:22:57.800 INFO:teuthology.orchestra.run.smithi189.stdout:Install 4 Packages 2024-09-11T08:22:57.800 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:22:57.800 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 79 M 2024-09-11T08:22:57.800 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 126 M 2024-09-11T08:22:57.801 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:22:58.036 INFO:teuthology.orchestra.run.smithi189.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 195 kB/s | 23 kB 00:00 2024-09-11T08:22:58.714 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-11T08:22:58.744 INFO:teuthology.orchestra.run.smithi189.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-09-11T08:22:59.269 INFO:teuthology.orchestra.run.smithi189.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 23 MB/s | 29 MB 00:01 2024-09-11T08:22:59.670 INFO:teuthology.orchestra.run.smithi189.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 20 MB/s | 35 MB 00:01 2024-09-11T08:22:59.670 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:22:59.670 INFO:teuthology.orchestra.run.smithi189.stdout:Total 42 MB/s | 79 MB 00:01 2024-09-11T08:23:00.348 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:23:00.696 INFO:teuthology.orchestra.run.smithi131.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T08:23:00.696 INFO:teuthology.orchestra.run.smithi131.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T08:23:00.708 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:23:00.708 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:23:00.788 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:23:00.795 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:23:00.795 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-09-11T08:23:00.796 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:23:00.796 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:23:00.796 INFO:teuthology.orchestra.run.smithi131.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T08:23:00.796 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-11T08:23:00.796 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T08:23:00.796 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T08:23:00.796 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T08:23:00.796 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:23:00.796 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:23:00.796 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:23:00.796 INFO:teuthology.orchestra.run.smithi131.stdout:Install 4 Packages 2024-09-11T08:23:00.796 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:23:00.797 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 79 M 2024-09-11T08:23:00.797 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 126 M 2024-09-11T08:23:00.797 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:23:01.174 INFO:teuthology.orchestra.run.smithi131.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 231 kB/s | 23 kB 00:00 2024-09-11T08:23:01.557 INFO:teuthology.orchestra.run.smithi131.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-11T08:23:01.640 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:23:01.640 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:23:01.941 INFO:teuthology.orchestra.run.smithi131.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 37 MB/s | 29 MB 00:00 2024-09-11T08:23:02.258 INFO:teuthology.orchestra.run.smithi131.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 29 MB/s | 35 MB 00:01 2024-09-11T08:23:02.258 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:23:02.258 INFO:teuthology.orchestra.run.smithi131.stdout:Total 54 MB/s | 79 MB 00:01 2024-09-11T08:23:02.937 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:23:03.319 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:23:03.319 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:23:04.067 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:23:04.262 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:23:04.262 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:23:04.719 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T08:23:04.812 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:23:06.707 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:23:07.444 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T08:23:07.533 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:23:07.622 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:23:07.758 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:23:10.312 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:23:10.451 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:23:12.589 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:23:12.625 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:23:15.273 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:23:15.392 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:23:17.496 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:23:20.224 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:23:55.914 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:23:55.953 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:23:56.321 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:23:56.321 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T08:23:56.321 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:23:56.321 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:23:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:23:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:23:57.215 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:23:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T08:23:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T08:23:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T08:23:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T08:23:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:23:57.216 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:23:57.576 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T08:23:57.664 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T08:23:57.665 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T08:23:57.665 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T08:23:57.665 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T08:23:57.665 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T08:23:57.665 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-11T08:23:57.665 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi189.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T08:23:57.666 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T08:23:57.725 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T08:23:57.725 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T08:23:57.725 DEBUG:teuthology.orchestra.run.smithi189:> sudo rpm -qi grub2-tools 2024-09-11T08:23:57.757 INFO:teuthology.orchestra.run.smithi189.stdout:Name : grub2-tools 2024-09-11T08:23:57.758 INFO:teuthology.orchestra.run.smithi189.stdout:Epoch : 1 2024-09-11T08:23:57.758 INFO:teuthology.orchestra.run.smithi189.stdout:Version : 2.06 2024-09-11T08:23:57.758 INFO:teuthology.orchestra.run.smithi189.stdout:Release : 82.el9 2024-09-11T08:23:57.758 INFO:teuthology.orchestra.run.smithi189.stdout:Architecture: x86_64 2024-09-11T08:23:57.758 INFO:teuthology.orchestra.run.smithi189.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T08:23:57.758 INFO:teuthology.orchestra.run.smithi189.stdout:Group : Unspecified 2024-09-11T08:23:57.758 INFO:teuthology.orchestra.run.smithi189.stdout:Size : 8274670 2024-09-11T08:23:57.758 INFO:teuthology.orchestra.run.smithi189.stdout:License : GPLv3+ 2024-09-11T08:23:57.758 INFO:teuthology.orchestra.run.smithi189.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T08:23:57.758 INFO:teuthology.orchestra.run.smithi189.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T08:23:57.758 INFO:teuthology.orchestra.run.smithi189.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T08:23:57.758 INFO:teuthology.orchestra.run.smithi189.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T08:23:57.758 INFO:teuthology.orchestra.run.smithi189.stdout:Packager : builder@centos.org 2024-09-11T08:23:57.758 INFO:teuthology.orchestra.run.smithi189.stdout:Vendor : CentOS 2024-09-11T08:23:57.759 INFO:teuthology.orchestra.run.smithi189.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T08:23:57.759 INFO:teuthology.orchestra.run.smithi189.stdout:Summary : Support tools for GRUB. 2024-09-11T08:23:57.759 INFO:teuthology.orchestra.run.smithi189.stdout:Description : 2024-09-11T08:23:57.759 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:23:57.759 INFO:teuthology.orchestra.run.smithi189.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T08:23:57.759 INFO:teuthology.orchestra.run.smithi189.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T08:23:57.759 INFO:teuthology.orchestra.run.smithi189.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T08:23:57.759 INFO:teuthology.orchestra.run.smithi189.stdout:hardware devices. 2024-09-11T08:23:57.759 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:23:57.759 INFO:teuthology.orchestra.run.smithi189.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T08:23:57.760 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T08:23:57.760 INFO:teuthology.task.kernel:Updating grub on smithi189 to boot 5.14.0-505.el9.x86_64 2024-09-11T08:23:57.760 DEBUG:teuthology.orchestra.run.smithi189:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T08:23:58.167 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:23:58.207 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:23:58.490 INFO:teuthology.orchestra.run.smithi189.stderr:Generating grub configuration file ... 2024-09-11T08:23:58.570 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:23:58.570 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T08:23:58.570 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T08:23:58.570 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T08:23:59.421 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T08:23:59.421 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:23:59.421 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:23:59.421 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T08:23:59.421 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T08:23:59.421 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T08:23:59.421 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T08:23:59.421 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:23:59.421 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:23:59.789 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T08:23:59.847 INFO:teuthology.orchestra.run.smithi189.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T08:23:59.877 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T08:23:59.878 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T08:23:59.878 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T08:23:59.878 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T08:23:59.878 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T08:23:59.878 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T08:23:59.878 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi131.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T08:23:59.878 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T08:23:59.885 INFO:teuthology.orchestra.run.smithi189.stderr:done 2024-09-11T08:23:59.888 DEBUG:teuthology.orchestra.run.smithi189:> mktemp 2024-09-11T08:23:59.904 INFO:teuthology.orchestra.run.smithi189.stdout:/tmp/tmp.KvgKp2bOVH 2024-09-11T08:23:59.904 DEBUG:teuthology.orchestra.run.smithi189:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.KvgKp2bOVH 2024-09-11T08:23:59.937 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T08:23:59.938 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T08:23:59.938 DEBUG:teuthology.orchestra.run.smithi131:> sudo rpm -qi grub2-tools 2024-09-11T08:23:59.970 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0666 /tmp/tmp.KvgKp2bOVH 2024-09-11T08:23:59.978 INFO:teuthology.orchestra.run.smithi131.stdout:Name : grub2-tools 2024-09-11T08:23:59.978 INFO:teuthology.orchestra.run.smithi131.stdout:Epoch : 1 2024-09-11T08:23:59.978 INFO:teuthology.orchestra.run.smithi131.stdout:Version : 2.06 2024-09-11T08:23:59.978 INFO:teuthology.orchestra.run.smithi131.stdout:Release : 82.el9 2024-09-11T08:23:59.978 INFO:teuthology.orchestra.run.smithi131.stdout:Architecture: x86_64 2024-09-11T08:23:59.978 INFO:teuthology.orchestra.run.smithi131.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T08:23:59.978 INFO:teuthology.orchestra.run.smithi131.stdout:Group : Unspecified 2024-09-11T08:23:59.978 INFO:teuthology.orchestra.run.smithi131.stdout:Size : 8274670 2024-09-11T08:23:59.978 INFO:teuthology.orchestra.run.smithi131.stdout:License : GPLv3+ 2024-09-11T08:23:59.979 INFO:teuthology.orchestra.run.smithi131.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T08:23:59.979 INFO:teuthology.orchestra.run.smithi131.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T08:23:59.979 INFO:teuthology.orchestra.run.smithi131.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T08:23:59.979 INFO:teuthology.orchestra.run.smithi131.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T08:23:59.979 INFO:teuthology.orchestra.run.smithi131.stdout:Packager : builder@centos.org 2024-09-11T08:23:59.979 INFO:teuthology.orchestra.run.smithi131.stdout:Vendor : CentOS 2024-09-11T08:23:59.979 INFO:teuthology.orchestra.run.smithi131.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T08:23:59.979 INFO:teuthology.orchestra.run.smithi131.stdout:Summary : Support tools for GRUB. 2024-09-11T08:23:59.979 INFO:teuthology.orchestra.run.smithi131.stdout:Description : 2024-09-11T08:23:59.979 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:23:59.979 INFO:teuthology.orchestra.run.smithi131.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T08:23:59.979 INFO:teuthology.orchestra.run.smithi131.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T08:23:59.979 INFO:teuthology.orchestra.run.smithi131.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T08:23:59.980 INFO:teuthology.orchestra.run.smithi131.stdout:hardware devices. 2024-09-11T08:23:59.980 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:23:59.980 INFO:teuthology.orchestra.run.smithi131.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T08:23:59.981 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T08:23:59.981 INFO:teuthology.task.kernel:Updating grub on smithi131 to boot 5.14.0-505.el9.x86_64 2024-09-11T08:23:59.981 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T08:24:00.116 DEBUG:teuthology.orchestra.remote:smithi189:/tmp/tmp.KvgKp2bOVH is 6KB 2024-09-11T08:24:00.166 DEBUG:teuthology.orchestra.run.smithi189:> rm -fr /tmp/tmp.KvgKp2bOVH 2024-09-11T08:24:00.181 DEBUG:teuthology.orchestra.run.smithi189:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T08:24:00.244 INFO:teuthology.orchestra.run.smithi189.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T08:24:00.244 INFO:teuthology.orchestra.run.smithi189.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T08:24:00.244 INFO:teuthology.orchestra.run.smithi189.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T08:24:00.244 INFO:teuthology.orchestra.run.smithi189.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T08:24:00.245 DEBUG:teuthology.orchestra.run.smithi189:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T08:24:00.376 DEBUG:teuthology.orchestra.run.smithi189:> sudo shutdown -r now 2024-09-11T08:24:00.629 INFO:teuthology.orchestra.run.smithi131.stderr:Generating grub configuration file ... 2024-09-11T08:24:02.016 INFO:teuthology.orchestra.run.smithi131.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T08:24:02.050 INFO:teuthology.orchestra.run.smithi131.stderr:done 2024-09-11T08:24:02.052 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-09-11T08:24:02.068 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.Le5xJ0hJgh 2024-09-11T08:24:02.069 DEBUG:teuthology.orchestra.run.smithi131:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Le5xJ0hJgh 2024-09-11T08:24:02.147 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.Le5xJ0hJgh 2024-09-11T08:24:02.297 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.Le5xJ0hJgh is 6KB 2024-09-11T08:24:02.318 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.Le5xJ0hJgh 2024-09-11T08:24:02.333 DEBUG:teuthology.orchestra.run.smithi131:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T08:24:02.395 INFO:teuthology.orchestra.run.smithi131.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T08:24:02.395 INFO:teuthology.orchestra.run.smithi131.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T08:24:02.395 INFO:teuthology.orchestra.run.smithi131.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T08:24:02.395 INFO:teuthology.orchestra.run.smithi131.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T08:24:02.397 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T08:24:02.534 DEBUG:teuthology.orchestra.run.smithi131:> sudo shutdown -r now 2024-09-11T08:24:30.406 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-11T08:24:30.407 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-11T08:24:30.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:24:32.538 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T08:24:32.538 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-11T08:24:32.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:24:48.961 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-09-11T08:24:57.969 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-11T08:24:57.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:25:01.021 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-09-11T08:25:13.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-11T08:25:13.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:25:31.457 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-09-11T08:25:32.542 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T08:25:41.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-11T08:25:41.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:25:41.833 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-09-11T08:25:42.304 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-11T08:25:42.305 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-11T08:25:42.305 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-09-11T08:25:42.320 INFO:teuthology.orchestra.run.smithi131.stdout:5.14.0-505.el9.x86_64 2024-09-11T08:25:42.320 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T08:25:42.320 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T08:25:42.321 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T08:25:43.321 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T08:25:43.322 DEBUG:teuthology.orchestra.run.smithi131:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T08:25:43.377 INFO:teuthology.orchestra.run.smithi131.stdout:ttyS1 2024-09-11T08:25:43.404 DEBUG:teuthology.parallel:result is None 2024-09-11T08:25:46.459 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-11T08:25:46.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:25:47.156 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-09-11T08:25:47.584 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-11T08:25:47.584 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-11T08:25:47.584 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-09-11T08:25:47.599 INFO:teuthology.orchestra.run.smithi189.stdout:5.14.0-505.el9.x86_64 2024-09-11T08:25:47.599 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T08:25:47.599 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T08:25:47.599 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T08:25:48.600 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-11T08:25:48.600 DEBUG:teuthology.orchestra.run.smithi189:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T08:25:48.669 INFO:teuthology.orchestra.run.smithi189.stdout:ttyS1 2024-09-11T08:25:48.693 DEBUG:teuthology.parallel:result is None 2024-09-11T08:25:48.693 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T08:25:48.701 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T08:25:48.701 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T08:25:48.704 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T08:25:48.720 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T08:25:48.728 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T08:25:48.743 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T08:25:48.744 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T08:25:48.763 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T08:25:48.808 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T08:25:48.817 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T08:25:48.817 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T08:25:48.845 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T08:25:48.875 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T08:25:48.877 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T08:25:48.886 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T08:25:48.889 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T08:25:48.892 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T08:25:48.899 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T08:25:48.900 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T08:25:48.935 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T08:25:48.962 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T08:25:48.972 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T08:25:48.972 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T08:25:49.002 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T08:25:49.017 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T08:25:49.099 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T08:25:49.162 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T08:25:49.206 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T08:25:49.245 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-11T08:25:49.246 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T08:25:49.310 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T08:25:49.352 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T08:25:49.403 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T08:25:49.480 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T08:25:49.526 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-11T08:25:49.526 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T08:25:49.598 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2024-09-11T08:25:49.601 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2024-09-11T08:25:49.642 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:25:49.685 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:25:50.059 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T08:25:50.068 INFO:teuthology.task.internal:Starting timer... 2024-09-11T08:25:50.068 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T08:25:50.145 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T08:25:50.229 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-11T08:25:50.230 DEBUG:teuthology.orchestra.run.smithi131:> sudo service auditd rotate 2024-09-11T08:25:50.278 INFO:teuthology.orchestra.run.smithi131.stdout:Rotating logs: 2024-09-11T08:25:50.279 DEBUG:teuthology.orchestra.run.smithi189:> sudo service auditd rotate 2024-09-11T08:25:50.353 INFO:teuthology.orchestra.run.smithi189.stdout:Rotating logs: 2024-09-11T08:25:50.355 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T08:25:50.355 DEBUG:teuthology.orchestra.run.smithi131:> /usr/sbin/getenforce 2024-09-11T08:25:50.385 INFO:teuthology.orchestra.run.smithi131.stdout:Permissive 2024-09-11T08:25:50.385 DEBUG:teuthology.orchestra.run.smithi189:> /usr/sbin/getenforce 2024-09-11T08:25:50.404 INFO:teuthology.orchestra.run.smithi189.stdout:Permissive 2024-09-11T08:25:50.404 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi131.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi189.front.sepia.ceph.com': 'permissive'} 2024-09-11T08:25:50.405 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:25:50.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:25:50.454 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:25:50.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:25:50.480 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T08:25:50.480 DEBUG:teuthology.orchestra.run.smithi131:> sudo /usr/sbin/setenforce permissive 2024-09-11T08:25:50.528 DEBUG:teuthology.orchestra.run.smithi189:> sudo /usr/sbin/setenforce permissive 2024-09-11T08:25:50.573 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T08:25:50.582 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T08:25:50.594 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T08:25:50.594 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi131.front.sepia.ceph.com,smithi189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T08:29:40.437 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi131.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com')] 2024-09-11T08:29:40.439 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-11T08:29:40.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:29:40.520 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-09-11T08:29:40.605 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-11T08:29:40.605 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-11T08:29:40.605 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T08:29:40.690 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-09-11T08:29:40.776 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-11T08:29:40.777 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T08:29:40.788 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T08:29:40.788 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T08:29:40.789 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:29:40.792 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T08:29:40.792 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:29:40.819 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T08:29:40.834 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T08:29:40.846 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T08:29:40.858 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: ntpd: command not found 2024-09-11T08:29:40.859 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T08:29:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:506 Cannot talk to daemon 2024-09-11T08:29:40.884 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T08:29:40.884 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: ntpd: command not found 2024-09-11T08:29:40.895 INFO:teuthology.orchestra.run.smithi189.stdout:506 Cannot talk to daemon 2024-09-11T08:29:40.897 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T08:29:40.909 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T08:29:40.923 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T08:29:40.955 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:29:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:29:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-09-11T08:29:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:29:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:29:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:29:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:29:40.979 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:29:40.982 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:29:40.982 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-09-11T08:29:40.983 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:29:40.983 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:29:40.983 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:29:40.983 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:29:40.983 INFO:teuthology.run_tasks:Running task pexec... 2024-09-11T08:29:40.993 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-11T08:29:40.993 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi131.front.sepia.ceph.com 2024-09-11T08:29:40.994 DEBUG:teuthology.orchestra.run.smithi131:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T08:29:40.994 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi189.front.sepia.ceph.com 2024-09-11T08:29:40.994 DEBUG:teuthology.orchestra.run.smithi189:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T08:29:41.347 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:29:41.349 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:29:41.349 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-09-11T08:29:41.349 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:29:41.349 INFO:teuthology.orchestra.run.smithi131.stdout:Removing: 2024-09-11T08:29:41.350 INFO:teuthology.orchestra.run.smithi131.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T08:29:41.350 INFO:teuthology.orchestra.run.smithi131.stdout:Removing unused dependencies: 2024-09-11T08:29:41.350 INFO:teuthology.orchestra.run.smithi131.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T08:29:41.350 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:29:41.350 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:29:41.350 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:29:41.350 INFO:teuthology.orchestra.run.smithi131.stdout:Remove 2 Packages 2024-09-11T08:29:41.350 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:29:41.350 INFO:teuthology.orchestra.run.smithi131.stdout:Freed space: 5.5 M 2024-09-11T08:29:41.350 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:29:41.354 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:29:41.354 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:29:41.375 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:29:41.385 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:29:41.385 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-09-11T08:29:41.385 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:29:41.385 INFO:teuthology.orchestra.run.smithi189.stdout:Removing: 2024-09-11T08:29:41.385 INFO:teuthology.orchestra.run.smithi189.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T08:29:41.385 INFO:teuthology.orchestra.run.smithi189.stdout:Removing unused dependencies: 2024-09-11T08:29:41.385 INFO:teuthology.orchestra.run.smithi189.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T08:29:41.385 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:29:41.385 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:29:41.385 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:29:41.385 INFO:teuthology.orchestra.run.smithi189.stdout:Remove 2 Packages 2024-09-11T08:29:41.386 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:29:41.386 INFO:teuthology.orchestra.run.smithi189.stdout:Freed space: 5.5 M 2024-09-11T08:29:41.386 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:29:41.389 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:29:41.389 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:29:41.402 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:29:41.403 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:29:41.445 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:29:41.445 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:29:41.575 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:29:41.600 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:29:41.644 INFO:teuthology.orchestra.run.smithi131.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T08:29:41.663 INFO:teuthology.orchestra.run.smithi189.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T08:29:41.698 INFO:teuthology.orchestra.run.smithi131.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T08:29:41.713 INFO:teuthology.orchestra.run.smithi189.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T08:29:42.176 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T08:29:42.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T08:29:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T08:29:42.206 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T08:29:42.398 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T08:29:42.398 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:29:42.398 INFO:teuthology.orchestra.run.smithi189.stdout:Removed: 2024-09-11T08:29:42.398 INFO:teuthology.orchestra.run.smithi189.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T08:29:42.398 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:29:42.398 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:29:42.466 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T08:29:42.467 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:29:42.467 INFO:teuthology.orchestra.run.smithi131.stdout:Removed: 2024-09-11T08:29:42.467 INFO:teuthology.orchestra.run.smithi131.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T08:29:42.467 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:29:42.467 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:29:43.252 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:57 ago on Wed 11 Sep 2024 08:28:46 AM UTC. 2024-09-11T08:29:43.256 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:47 ago on Wed 11 Sep 2024 08:28:56 AM UTC. 2024-09-11T08:29:43.389 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:29:43.389 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:29:43.389 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-09-11T08:29:43.390 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:29:43.390 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:29:43.390 INFO:teuthology.orchestra.run.smithi131.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T08:29:43.390 INFO:teuthology.orchestra.run.smithi131.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T08:29:43.390 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-11T08:29:43.390 INFO:teuthology.orchestra.run.smithi131.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T08:29:43.390 INFO:teuthology.orchestra.run.smithi131.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T08:29:43.390 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T08:29:43.390 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T08:29:43.390 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:29:43.390 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:29:43.390 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:29:43.390 INFO:teuthology.orchestra.run.smithi131.stdout:Install 6 Packages 2024-09-11T08:29:43.391 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:29:43.391 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:29:43.391 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:29:43.391 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-09-11T08:29:43.391 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:29:43.391 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:29:43.392 INFO:teuthology.orchestra.run.smithi189.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T08:29:43.392 INFO:teuthology.orchestra.run.smithi189.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T08:29:43.392 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-11T08:29:43.392 INFO:teuthology.orchestra.run.smithi189.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T08:29:43.392 INFO:teuthology.orchestra.run.smithi189.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T08:29:43.392 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T08:29:43.392 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T08:29:43.392 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:29:43.392 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:29:43.392 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:29:43.392 INFO:teuthology.orchestra.run.smithi189.stdout:Install 6 Packages 2024-09-11T08:29:43.392 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:29:43.400 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 2.0 M 2024-09-11T08:29:43.401 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 9.2 M 2024-09-11T08:29:43.401 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:29:43.406 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 2.0 M 2024-09-11T08:29:43.406 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 9.2 M 2024-09-11T08:29:43.406 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:29:43.538 INFO:teuthology.orchestra.run.smithi131.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 413 kB/s | 44 kB 00:00 2024-09-11T08:29:43.544 INFO:teuthology.orchestra.run.smithi189.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 369 kB/s | 44 kB 00:00 2024-09-11T08:29:43.569 INFO:teuthology.orchestra.run.smithi189.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 728 kB/s | 106 kB 00:00 2024-09-11T08:29:43.588 INFO:teuthology.orchestra.run.smithi131.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 672 kB/s | 106 kB 00:00 2024-09-11T08:29:43.605 INFO:teuthology.orchestra.run.smithi131.stdout:(3/6): python3-configshell-1.1.30-1.el9.noarch. 1.0 MB/s | 72 kB 00:00 2024-09-11T08:29:43.611 INFO:teuthology.orchestra.run.smithi189.stdout:(3/6): python3-configshell-1.1.30-1.el9.noarch. 1.0 MB/s | 72 kB 00:00 2024-09-11T08:29:43.636 INFO:teuthology.orchestra.run.smithi189.stdout:(4/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.2 MB/s | 84 kB 00:00 2024-09-11T08:29:43.639 INFO:teuthology.orchestra.run.smithi131.stdout:(4/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-11T08:29:43.662 INFO:teuthology.orchestra.run.smithi189.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.7 MB/s | 906 kB 00:00 2024-09-11T08:29:43.722 INFO:teuthology.orchestra.run.smithi131.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.0 MB/s | 906 kB 00:00 2024-09-11T08:29:43.762 INFO:teuthology.orchestra.run.smithi189.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 5.4 MB/s | 837 kB 00:00 2024-09-11T08:29:43.762 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:29:43.762 INFO:teuthology.orchestra.run.smithi189.stdout:Total 5.5 MB/s | 2.0 MB 00:00 2024-09-11T08:29:43.798 INFO:teuthology.orchestra.run.smithi131.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.3 MB/s | 837 kB 00:00 2024-09-11T08:29:43.798 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:29:43.798 INFO:teuthology.orchestra.run.smithi131.stdout:Total 5.1 MB/s | 2.0 MB 00:00 2024-09-11T08:29:43.836 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:29:43.850 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:29:43.850 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:29:43.871 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:29:43.885 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:29:43.885 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:29:44.161 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:29:44.161 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:29:44.175 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:29:44.175 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:29:44.592 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:29:44.612 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:29:44.691 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-11T08:29:44.692 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-11T08:29:44.734 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-11T08:29:44.734 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-11T08:29:44.763 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-11T08:29:44.763 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-11T08:29:44.971 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-11T08:29:44.971 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-11T08:29:45.002 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T08:29:45.011 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T08:29:45.434 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T08:29:45.463 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T08:29:45.495 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T08:29:45.531 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T08:29:46.436 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T08:29:46.436 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-11T08:29:46.436 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-11T08:29:46.436 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-11T08:29:46.436 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-11T08:29:46.436 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-11T08:29:46.505 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T08:29:46.505 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-11T08:29:46.505 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-11T08:29:46.505 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-11T08:29:46.505 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-11T08:29:46.506 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-11T08:29:46.802 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-11T08:29:46.802 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:29:46.802 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:29:46.803 INFO:teuthology.orchestra.run.smithi189.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T08:29:46.803 INFO:teuthology.orchestra.run.smithi189.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T08:29:46.803 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T08:29:46.803 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:29:46.803 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:29:46.851 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-11T08:29:46.851 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:29:46.851 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:29:46.851 INFO:teuthology.orchestra.run.smithi131.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T08:29:46.851 INFO:teuthology.orchestra.run.smithi131.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T08:29:46.851 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T08:29:46.851 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:29:46.852 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:29:47.090 DEBUG:teuthology.parallel:result is None 2024-09-11T08:29:47.134 DEBUG:teuthology.parallel:result is None 2024-09-11T08:29:47.135 INFO:teuthology.run_tasks:Running task install... 2024-09-11T08:29:47.143 DEBUG:teuthology.task.install:project ceph 2024-09-11T08:29:47.143 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'}, 'extra_packages': ['cephadm']} 2024-09-11T08:29:47.144 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'} 2024-09-11T08:29:47.144 INFO:teuthology.task.install:Using flavor: default 2024-09-11T08:29:47.149 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T08:29:47.149 INFO:teuthology.task.install:extra packages: [] 2024-09-11T08:29:47.149 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'tag': None, 'wait_for_package': False} 2024-09-11T08:29:47.150 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:29:47.152 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'tag': None, 'wait_for_package': False} 2024-09-11T08:29:47.152 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:29:47.294 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/ 2024-09-11T08:29:47.294 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1623.gd0e6828a 2024-09-11T08:29:47.310 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/ 2024-09-11T08:29:47.311 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1623.gd0e6828a 2024-09-11T08:29:47.429 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T08:29:47.429 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-11T08:29:47.429 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T08:29:47.447 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T08:29:47.447 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-11T08:29:47.447 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T08:29:47.459 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T08:29:47.459 DEBUG:teuthology.orchestra.run.smithi131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e6828a2016d48cf25ad84064e50742bb1c39b9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T08:29:47.476 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T08:29:47.476 DEBUG:teuthology.orchestra.run.smithi189:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e6828a2016d48cf25ad84064e50742bb1c39b9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T08:29:47.534 DEBUG:teuthology.orchestra.run.smithi131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T08:29:47.541 DEBUG:teuthology.orchestra.run.smithi189:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T08:29:47.610 DEBUG:teuthology.orchestra.run.smithi131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T08:29:47.638 DEBUG:teuthology.orchestra.run.smithi189:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T08:29:47.661 INFO:teuthology.orchestra.run.smithi189.stdout:check_obsoletes = 1 2024-09-11T08:29:47.663 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum clean all 2024-09-11T08:29:47.674 INFO:teuthology.orchestra.run.smithi131.stdout:check_obsoletes = 1 2024-09-11T08:29:47.675 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum clean all 2024-09-11T08:29:47.882 INFO:teuthology.orchestra.run.smithi189.stdout:45 files removed 2024-09-11T08:29:47.901 INFO:teuthology.orchestra.run.smithi131.stdout:45 files removed 2024-09-11T08:29:47.909 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-radosgw 2024-09-11T08:29:47.928 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-radosgw 2024-09-11T08:29:48.460 INFO:teuthology.orchestra.run.smithi189.stdout:ceph packages for x86_64 292 kB/s | 77 kB 00:00 2024-09-11T08:29:48.487 INFO:teuthology.orchestra.run.smithi131.stdout:ceph packages for x86_64 283 kB/s | 77 kB 00:00 2024-09-11T08:29:48.744 INFO:teuthology.orchestra.run.smithi189.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-11T08:29:48.763 INFO:teuthology.orchestra.run.smithi131.stdout:ceph noarch packages 46 kB/s | 11 kB 00:00 2024-09-11T08:29:49.011 INFO:teuthology.orchestra.run.smithi189.stdout:ceph source packages 7.7 kB/s | 1.8 kB 00:00 2024-09-11T08:29:49.030 INFO:teuthology.orchestra.run.smithi131.stdout:ceph source packages 7.7 kB/s | 1.8 kB 00:00 2024-09-11T08:29:49.503 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.2 MB 00:00 2024-09-11T08:29:49.539 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-11T08:29:51.279 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-11T08:29:53.706 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 20 MB 00:03 2024-09-11T08:29:56.955 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-11T08:29:57.998 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-11T08:29:58.472 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 19 kB 00:00 2024-09-11T08:29:59.335 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-11T08:29:59.874 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-11T08:30:00.762 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-11T08:30:04.699 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-11T08:30:06.168 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-11T08:30:07.167 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:30:07.169 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:30:07.169 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-11T08:30:07.169 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:30:07.169 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:30:07.169 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 14 M 2024-09-11T08:30:07.169 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrading: 2024-09-11T08:30:07.169 INFO:teuthology.orchestra.run.smithi189.stdout: librados2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.1 M 2024-09-11T08:30:07.169 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T08:30:07.169 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-11T08:30:07.170 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 5.0 M 2024-09-11T08:30:07.170 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 23 M 2024-09-11T08:30:07.170 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T08:30:07.170 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T08:30:07.170 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 636 k 2024-09-11T08:30:07.170 INFO:teuthology.orchestra.run.smithi189.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T08:30:07.170 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T08:30:07.170 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T08:30:07.170 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 457 k 2024-09-11T08:30:07.170 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T08:30:07.170 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 4.9 M 2024-09-11T08:30:07.170 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T08:30:07.170 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T08:30:07.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 45 k 2024-09-11T08:30:07.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 120 k 2024-09-11T08:30:07.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 156 k 2024-09-11T08:30:07.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T08:30:07.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 320 k 2024-09-11T08:30:07.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 294 k 2024-09-11T08:30:07.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 99 k 2024-09-11T08:30:07.171 INFO:teuthology.orchestra.run.smithi189.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T08:30:07.171 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:07.171 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:30:07.171 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:30:07.171 INFO:teuthology.orchestra.run.smithi189.stdout:Install 22 Packages 2024-09-11T08:30:07.171 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrade 2 Packages 2024-09-11T08:30:07.171 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:07.172 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 58 M 2024-09-11T08:30:07.172 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:30:08.050 INFO:teuthology.orchestra.run.smithi189.stdout:(1/24): ceph-base-17.2.7-1623.gd0e6828a.el9.x86 7.3 MB/s | 5.0 MB 00:00 2024-09-11T08:30:08.117 INFO:teuthology.orchestra.run.smithi189.stdout:(2/24): ceph-selinux-17.2.7-1623.gd0e6828a.el9. 377 kB/s | 25 kB 00:00 2024-09-11T08:30:08.276 INFO:teuthology.orchestra.run.smithi189.stdout:(3/24): ceph-radosgw-17.2.7-1623.gd0e6828a.el9. 16 MB/s | 14 MB 00:00 2024-09-11T08:30:08.309 INFO:teuthology.orchestra.run.smithi189.stdout:(4/24): libcephfs2-17.2.7-1623.gd0e6828a.el9.x8 3.2 MB/s | 636 kB 00:00 2024-09-11T08:30:08.334 INFO:teuthology.orchestra.run.smithi189.stdout:(5/24): libradosstriper1-17.2.7-1623.gd0e6828a. 7.7 MB/s | 457 kB 00:00 2024-09-11T08:30:08.385 INFO:teuthology.orchestra.run.smithi189.stdout:(6/24): python3-ceph-argparse-17.2.7-1623.gd0e6 903 kB/s | 45 kB 00:00 2024-09-11T08:30:08.460 INFO:teuthology.orchestra.run.smithi189.stdout:(7/24): python3-ceph-common-17.2.7-1623.gd0e682 1.6 MB/s | 120 kB 00:00 2024-09-11T08:30:08.548 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:30:08.550 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:30:08.550 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-11T08:30:08.550 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:30:08.550 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:30:08.550 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 14 M 2024-09-11T08:30:08.551 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrading: 2024-09-11T08:30:08.551 INFO:teuthology.orchestra.run.smithi131.stdout: librados2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.1 M 2024-09-11T08:30:08.551 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T08:30:08.551 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-11T08:30:08.551 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 5.0 M 2024-09-11T08:30:08.551 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 23 M 2024-09-11T08:30:08.551 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T08:30:08.551 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T08:30:08.551 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 636 k 2024-09-11T08:30:08.551 INFO:teuthology.orchestra.run.smithi131.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T08:30:08.551 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T08:30:08.551 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T08:30:08.552 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 457 k 2024-09-11T08:30:08.552 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T08:30:08.552 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 4.9 M 2024-09-11T08:30:08.552 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T08:30:08.552 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T08:30:08.552 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 45 k 2024-09-11T08:30:08.552 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 120 k 2024-09-11T08:30:08.552 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 156 k 2024-09-11T08:30:08.552 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T08:30:08.552 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 320 k 2024-09-11T08:30:08.552 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 294 k 2024-09-11T08:30:08.552 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 99 k 2024-09-11T08:30:08.553 INFO:teuthology.orchestra.run.smithi131.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T08:30:08.553 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:08.553 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:30:08.553 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:30:08.553 INFO:teuthology.orchestra.run.smithi131.stdout:Install 22 Packages 2024-09-11T08:30:08.553 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrade 2 Packages 2024-09-11T08:30:08.553 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:08.554 INFO:teuthology.orchestra.run.smithi189.stdout:(8/24): librgw2-17.2.7-1623.gd0e6828a.el9.x86_6 20 MB/s | 4.9 MB 00:00 2024-09-11T08:30:08.554 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 58 M 2024-09-11T08:30:08.554 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:30:08.577 INFO:teuthology.orchestra.run.smithi189.stdout:(9/24): python3-cephfs-17.2.7-1623.gd0e6828a.el 1.3 MB/s | 156 kB 00:00 2024-09-11T08:30:08.602 INFO:teuthology.orchestra.run.smithi189.stdout:(10/24): python3-rados-17.2.7-1623.gd0e6828a.el 6.2 MB/s | 320 kB 00:00 2024-09-11T08:30:08.636 INFO:teuthology.orchestra.run.smithi189.stdout:(11/24): python3-rbd-17.2.7-1623.gd0e6828a.el9. 4.9 MB/s | 294 kB 00:00 2024-09-11T08:30:08.661 INFO:teuthology.orchestra.run.smithi189.stdout:(12/24): python3-rgw-17.2.7-1623.gd0e6828a.el9. 1.7 MB/s | 99 kB 00:00 2024-09-11T08:30:08.887 INFO:teuthology.orchestra.run.smithi189.stdout:(13/24): ceph-common-17.2.7-1623.gd0e6828a.el9. 15 MB/s | 23 MB 00:01 2024-09-11T08:30:08.970 INFO:teuthology.orchestra.run.smithi189.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 147 kB/s | 45 kB 00:00 2024-09-11T08:30:09.087 INFO:teuthology.orchestra.run.smithi189.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 355 kB/s | 160 kB 00:00 2024-09-11T08:30:09.112 INFO:teuthology.orchestra.run.smithi189.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-09-11T08:30:09.146 INFO:teuthology.orchestra.run.smithi189.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 723 kB/s | 42 kB 00:00 2024-09-11T08:30:09.180 INFO:teuthology.orchestra.run.smithi189.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.2 MB/s | 662 kB 00:00 2024-09-11T08:30:09.221 INFO:teuthology.orchestra.run.smithi189.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 665 kB/s | 49 kB 00:00 2024-09-11T08:30:09.247 INFO:teuthology.orchestra.run.smithi189.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.3 MB/s | 309 kB 00:00 2024-09-11T08:30:09.272 INFO:teuthology.orchestra.run.smithi189.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 796 kB/s | 67 kB 00:00 2024-09-11T08:30:09.323 INFO:teuthology.orchestra.run.smithi189.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-11T08:30:09.494 INFO:teuthology.orchestra.run.smithi131.stdout:(1/24): ceph-base-17.2.7-1623.gd0e6828a.el9.x86 6.6 MB/s | 5.0 MB 00:00 2024-09-11T08:30:09.603 INFO:teuthology.orchestra.run.smithi131.stdout:(2/24): ceph-selinux-17.2.7-1623.gd0e6828a.el9. 232 kB/s | 25 kB 00:00 2024-09-11T08:30:09.631 INFO:teuthology.orchestra.run.smithi189.stdout:(23/24): librados2-17.2.7-1623.gd0e6828a.el9.x8 8.0 MB/s | 3.1 MB 00:00 2024-09-11T08:30:09.748 INFO:teuthology.orchestra.run.smithi189.stdout:(24/24): librbd1-17.2.7-1623.gd0e6828a.el9.x86_ 6.2 MB/s | 2.9 MB 00:00 2024-09-11T08:30:09.753 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:09.753 INFO:teuthology.orchestra.run.smithi189.stdout:Total 23 MB/s | 58 MB 00:02 2024-09-11T08:30:09.795 INFO:teuthology.orchestra.run.smithi131.stdout:(3/24): ceph-radosgw-17.2.7-1623.gd0e6828a.el9. 14 MB/s | 14 MB 00:01 2024-09-11T08:30:09.819 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:30:09.837 INFO:teuthology.orchestra.run.smithi131.stdout:(4/24): libcephfs2-17.2.7-1623.gd0e6828a.el9.x8 2.7 MB/s | 636 kB 00:00 2024-09-11T08:30:09.853 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:30:09.853 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:30:09.862 INFO:teuthology.orchestra.run.smithi131.stdout:(5/24): libradosstriper1-17.2.7-1623.gd0e6828a. 6.7 MB/s | 457 kB 00:00 2024-09-11T08:30:09.912 INFO:teuthology.orchestra.run.smithi131.stdout:(6/24): python3-ceph-argparse-17.2.7-1623.gd0e6 907 kB/s | 45 kB 00:00 2024-09-11T08:30:10.146 INFO:teuthology.orchestra.run.smithi131.stdout:(7/24): ceph-common-17.2.7-1623.gd0e6828a.el9.x 16 MB/s | 23 MB 00:01 2024-09-11T08:30:10.171 INFO:teuthology.orchestra.run.smithi131.stdout:(8/24): python3-ceph-common-17.2.7-1623.gd0e682 465 kB/s | 120 kB 00:00 2024-09-11T08:30:10.196 INFO:teuthology.orchestra.run.smithi131.stdout:(9/24): python3-cephfs-17.2.7-1623.gd0e6828a.el 3.1 MB/s | 156 kB 00:00 2024-09-11T08:30:10.230 INFO:teuthology.orchestra.run.smithi131.stdout:(10/24): python3-rados-17.2.7-1623.gd0e6828a.el 5.3 MB/s | 320 kB 00:00 2024-09-11T08:30:10.264 INFO:teuthology.orchestra.run.smithi131.stdout:(11/24): python3-rbd-17.2.7-1623.gd0e6828a.el9. 4.3 MB/s | 294 kB 00:00 2024-09-11T08:30:10.289 INFO:teuthology.orchestra.run.smithi131.stdout:(12/24): python3-rgw-17.2.7-1623.gd0e6828a.el9. 1.7 MB/s | 99 kB 00:00 2024-09-11T08:30:10.321 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:30:10.322 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:30:10.356 INFO:teuthology.orchestra.run.smithi131.stdout:(13/24): librgw2-17.2.7-1623.gd0e6828a.el9.x86_ 9.5 MB/s | 4.9 MB 00:00 2024-09-11T08:30:10.465 INFO:teuthology.orchestra.run.smithi131.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 258 kB/s | 45 kB 00:00 2024-09-11T08:30:10.499 INFO:teuthology.orchestra.run.smithi131.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 684 kB/s | 160 kB 00:00 2024-09-11T08:30:10.557 INFO:teuthology.orchestra.run.smithi131.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 728 kB/s | 42 kB 00:00 2024-09-11T08:30:10.591 INFO:teuthology.orchestra.run.smithi131.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-11T08:30:10.641 INFO:teuthology.orchestra.run.smithi131.stdout:(18/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 3.6 MB/s | 309 kB 00:00 2024-09-11T08:30:10.666 INFO:teuthology.orchestra.run.smithi131.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 2.9 MB/s | 67 kB 00:00 2024-09-11T08:30:10.691 INFO:teuthology.orchestra.run.smithi131.stdout:(20/24): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-09-11T08:30:10.726 INFO:teuthology.orchestra.run.smithi131.stdout:(21/24): librdkafka-1.6.1-102.el9.x86_64.rpm 1.8 MB/s | 662 kB 00:00 2024-09-11T08:30:10.792 INFO:teuthology.orchestra.run.smithi131.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-11T08:30:10.918 INFO:teuthology.orchestra.run.smithi131.stdout:(23/24): librados2-17.2.7-1623.gd0e6828a.el9.x8 14 MB/s | 3.1 MB 00:00 2024-09-11T08:30:10.976 INFO:teuthology.orchestra.run.smithi131.stdout:(24/24): librbd1-17.2.7-1623.gd0e6828a.el9.x86_ 12 MB/s | 2.9 MB 00:00 2024-09-11T08:30:10.981 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:10.981 INFO:teuthology.orchestra.run.smithi131.stdout:Total 24 MB/s | 58 MB 00:02 2024-09-11T08:30:11.043 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:30:11.077 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:30:11.077 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:30:11.521 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:30:11.533 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:30:11.533 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:30:11.920 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-11T08:30:11.992 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:30:12.084 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:30:12.161 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T08:30:12.193 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:30:12.281 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:30:12.332 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-11T08:30:12.365 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-11T08:30:12.745 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:30:12.811 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-11T08:30:12.838 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T08:30:12.900 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T08:30:12.929 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-11T08:30:12.972 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 10/26 2024-09-11T08:30:13.008 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T08:30:13.071 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T08:30:13.099 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T08:30:13.116 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-11T08:30:13.148 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T08:30:13.188 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-11T08:30:13.232 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-11T08:30:13.237 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:30:13.273 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-11T08:30:13.330 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:30:13.417 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T08:30:13.447 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:30:13.536 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:30:13.567 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-11T08:30:13.591 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-11T08:30:13.602 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T08:30:13.619 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-11T08:30:13.684 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T08:30:13.757 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 19/26 2024-09-11T08:30:13.805 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 20/26 2024-09-11T08:30:14.081 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-11T08:30:14.109 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T08:30:14.172 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T08:30:14.209 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-11T08:30:14.251 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 10/26 2024-09-11T08:30:14.288 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T08:30:14.351 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T08:30:14.378 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T08:30:14.452 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T08:30:14.494 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-11T08:30:14.529 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-11T08:30:14.569 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-11T08:30:14.870 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-11T08:30:14.916 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T08:30:15.005 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T08:30:15.079 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 19/26 2024-09-11T08:30:15.110 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 20/26 2024-09-11T08:30:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:30:16.326 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:30:16.855 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:30:16.887 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T08:30:16.995 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T08:30:16.995 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T08:30:16.995 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T08:30:16.995 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:17.030 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:30:17.522 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:30:17.557 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:30:18.080 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T08:30:18.118 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T08:30:18.203 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T08:30:18.203 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T08:30:18.203 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T08:30:18.203 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:18.245 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:30:26.786 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:30:26.786 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /sys 2024-09-11T08:30:26.786 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /proc 2024-09-11T08:30:26.786 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /mnt 2024-09-11T08:30:26.786 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /var/tmp 2024-09-11T08:30:26.786 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /home 2024-09-11T08:30:26.786 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /root 2024-09-11T08:30:26.786 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /tmp 2024-09-11T08:30:26.786 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:28.137 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T08:30:28.225 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T08:30:28.225 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:30:28.225 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T08:30:28.225 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T08:30:28.225 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T08:30:28.225 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:28.226 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T08:30:28.307 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T08:30:28.307 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:30:28.321 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:30:28.322 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /sys 2024-09-11T08:30:28.322 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /proc 2024-09-11T08:30:28.322 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /mnt 2024-09-11T08:30:28.322 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /var/tmp 2024-09-11T08:30:28.322 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /home 2024-09-11T08:30:28.322 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /root 2024-09-11T08:30:28.322 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /tmp 2024-09-11T08:30:28.322 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:29.573 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:30:29.573 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/26 2024-09-11T08:30:29.573 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:30:29.573 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T08:30:29.573 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:30:29.573 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/26 2024-09-11T08:30:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 6/26 2024-09-11T08:30:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/26 2024-09-11T08:30:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 8/26 2024-09-11T08:30:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 9/26 2024-09-11T08:30:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/26 2024-09-11T08:30:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T08:30:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T08:30:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 13/26 2024-09-11T08:30:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-11T08:30:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-11T08:30:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-11T08:30:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-11T08:30:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-11T08:30:29.575 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-11T08:30:29.575 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-11T08:30:29.575 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-11T08:30:29.575 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-11T08:30:29.575 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:30:29.575 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-11T08:30:29.575 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 25/26 2024-09-11T08:30:29.658 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T08:30:29.745 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T08:30:29.745 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:30:29.745 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T08:30:29.745 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T08:30:29.745 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T08:30:29.745 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:29.746 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T08:30:29.836 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T08:30:29.836 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:30:30.373 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:30:30.373 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:30.373 INFO:teuthology.orchestra.run.smithi189.stdout:Upgraded: 2024-09-11T08:30:30.373 INFO:teuthology.orchestra.run.smithi189.stdout: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:30.373 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:30.373 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:30:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T08:30:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T08:30:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T08:30:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T08:30:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T08:30:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:30.375 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T08:30:30.375 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T08:30:30.375 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:30.375 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:30.375 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:30.375 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T08:30:30.375 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:30.375 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:30.375 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:30.375 INFO:teuthology.orchestra.run.smithi189.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T08:30:30.375 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:30.376 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:30:30.767 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-test 2024-09-11T08:30:31.087 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:30:31.088 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/26 2024-09-11T08:30:31.088 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T08:30:31.088 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T08:30:31.088 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T08:30:31.088 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/26 2024-09-11T08:30:31.088 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 6/26 2024-09-11T08:30:31.088 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/26 2024-09-11T08:30:31.088 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 8/26 2024-09-11T08:30:31.088 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 9/26 2024-09-11T08:30:31.088 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/26 2024-09-11T08:30:31.088 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T08:30:31.088 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T08:30:31.089 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 13/26 2024-09-11T08:30:31.089 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-11T08:30:31.089 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-11T08:30:31.089 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-11T08:30:31.089 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-11T08:30:31.089 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-11T08:30:31.089 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-11T08:30:31.089 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-11T08:30:31.089 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-11T08:30:31.089 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-11T08:30:31.089 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T08:30:31.089 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-11T08:30:31.089 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 25/26 2024-09-11T08:30:31.337 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:27 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:30:31.475 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:30:31.475 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:30:31.475 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-11T08:30:31.475 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:30:31.475 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:30:31.476 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 48 M 2024-09-11T08:30:31.476 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-11T08:30:31.476 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 163 k 2024-09-11T08:30:31.476 INFO:teuthology.orchestra.run.smithi189.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T08:30:31.476 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T08:30:31.476 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:31.476 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:30:31.476 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:30:31.476 INFO:teuthology.orchestra.run.smithi189.stdout:Install 4 Packages 2024-09-11T08:30:31.476 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:31.477 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 48 M 2024-09-11T08:30:31.477 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 193 M 2024-09-11T08:30:31.477 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:30:31.768 INFO:teuthology.orchestra.run.smithi189.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.7 MB/s | 304 kB 00:00 2024-09-11T08:30:31.827 INFO:teuthology.orchestra.run.smithi189.stdout:(2/4): libcephsqlite-17.2.7-1623.gd0e6828a.el9. 694 kB/s | 163 kB 00:00 2024-09-11T08:30:31.852 INFO:teuthology.orchestra.run.smithi189.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 762 kB/s | 64 kB 00:00 2024-09-11T08:30:32.062 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T08:30:32.062 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:32.062 INFO:teuthology.orchestra.run.smithi131.stdout:Upgraded: 2024-09-11T08:30:32.062 INFO:teuthology.orchestra.run.smithi131.stdout: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:32.062 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:32.062 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:30:32.062 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:32.062 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:32.062 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:32.062 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:32.063 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T08:30:32.063 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:32.063 INFO:teuthology.orchestra.run.smithi131.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T08:30:32.063 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T08:30:32.063 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T08:30:32.063 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:32.063 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T08:30:32.063 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:32.063 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T08:30:32.063 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T08:30:32.063 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:32.063 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:32.063 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:32.063 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T08:30:32.063 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:32.064 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:32.064 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:32.064 INFO:teuthology.orchestra.run.smithi131.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T08:30:32.064 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:32.064 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:30:32.458 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-test 2024-09-11T08:30:33.024 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:27 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:30:33.110 INFO:teuthology.orchestra.run.smithi189.stdout:(4/4): ceph-test-17.2.7-1623.gd0e6828a.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-11T08:30:33.112 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:33.113 INFO:teuthology.orchestra.run.smithi189.stdout:Total 30 MB/s | 48 MB 00:01 2024-09-11T08:30:33.141 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:30:33.157 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:30:33.157 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:30:33.158 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-11T08:30:33.158 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:30:33.158 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:30:33.158 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 48 M 2024-09-11T08:30:33.158 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-11T08:30:33.158 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 163 k 2024-09-11T08:30:33.158 INFO:teuthology.orchestra.run.smithi131.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T08:30:33.158 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T08:30:33.158 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:33.158 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:30:33.158 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:30:33.159 INFO:teuthology.orchestra.run.smithi131.stdout:Install 4 Packages 2024-09-11T08:30:33.159 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:33.159 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 48 M 2024-09-11T08:30:33.159 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 193 M 2024-09-11T08:30:33.159 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:30:33.160 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:30:33.160 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:30:33.530 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:30:33.530 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:30:33.573 INFO:teuthology.orchestra.run.smithi131.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-11T08:30:33.615 INFO:teuthology.orchestra.run.smithi131.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-11T08:30:33.657 INFO:teuthology.orchestra.run.smithi131.stdout:(3/4): libcephsqlite-17.2.7-1623.gd0e6828a.el9. 751 kB/s | 163 kB 00:00 2024-09-11T08:30:34.052 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:30:34.143 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T08:30:34.184 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T08:30:34.221 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T08:30:34.959 INFO:teuthology.orchestra.run.smithi131.stdout:(4/4): ceph-test-17.2.7-1623.gd0e6828a.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-11T08:30:34.961 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:34.962 INFO:teuthology.orchestra.run.smithi131.stdout:Total 27 MB/s | 48 MB 00:01 2024-09-11T08:30:34.989 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:30:35.009 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:30:35.009 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:30:35.374 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:30:35.374 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:30:35.907 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:30:35.984 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T08:30:36.023 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T08:30:36.051 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T08:30:38.610 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T08:30:38.685 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T08:30:40.457 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T08:30:40.548 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T08:30:40.807 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T08:30:40.807 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/4 2024-09-11T08:30:40.807 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/4 2024-09-11T08:30:40.807 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T08:30:41.118 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T08:30:41.118 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:41.118 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:30:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T08:30:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T08:30:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:41.119 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:30:41.387 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph 2024-09-11T08:30:41.977 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:37 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:30:42.118 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:30:42.120 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:30:42.120 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-11T08:30:42.121 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:30:42.121 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:30:42.121 INFO:teuthology.orchestra.run.smithi189.stdout: ceph x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 6.5 k 2024-09-11T08:30:42.121 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-11T08:30:42.121 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.1 M 2024-09-11T08:30:42.121 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 1.4 M 2024-09-11T08:30:42.122 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 240 k 2024-09-11T08:30:42.122 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.9 M 2024-09-11T08:30:42.122 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 15 M 2024-09-11T08:30:42.122 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T08:30:42.122 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T08:30:42.122 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T08:30:42.122 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T08:30:42.122 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T08:30:42.122 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T08:30:42.122 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T08:30:42.122 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections 2024-09-11T08:30:42.122 INFO:teuthology.orchestra.run.smithi189.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-11T08:30:42.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T08:30:42.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T08:30:42.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T08:30:42.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T08:30:42.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T08:30:42.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T08:30:42.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T08:30:42.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T08:30:42.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T08:30:42.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T08:30:42.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T08:30:42.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T08:30:42.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T08:30:42.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T08:30:42.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T08:30:42.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T08:30:42.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T08:30:42.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T08:30:42.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T08:30:42.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T08:30:42.124 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:42.124 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:30:42.124 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:30:42.124 INFO:teuthology.orchestra.run.smithi189.stdout:Install 34 Packages 2024-09-11T08:30:42.124 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:42.125 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 27 M 2024-09-11T08:30:42.125 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 96 M 2024-09-11T08:30:42.125 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:30:42.589 INFO:teuthology.orchestra.run.smithi189.stdout:(1/34): ceph-17.2.7-1623.gd0e6828a.el9.x86_64.r 55 kB/s | 6.5 kB 00:00 2024-09-11T08:30:42.781 INFO:teuthology.orchestra.run.smithi189.stdout:(2/34): ceph-mgr-17.2.7-1623.gd0e6828a.el9.x86_ 4.5 MB/s | 1.4 MB 00:00 2024-09-11T08:30:42.823 INFO:teuthology.orchestra.run.smithi189.stdout:(3/34): ceph-mds-17.2.7-1623.gd0e6828a.el9.x86_ 5.9 MB/s | 2.1 MB 00:00 2024-09-11T08:30:42.907 INFO:teuthology.orchestra.run.smithi189.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1623.gd0e6 2.8 MB/s | 240 kB 00:00 2024-09-11T08:30:42.934 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T08:30:42.934 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/4 2024-09-11T08:30:42.934 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/4 2024-09-11T08:30:42.934 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T08:30:42.973 INFO:teuthology.orchestra.run.smithi189.stdout:(5/34): ceph-mon-17.2.7-1623.gd0e6828a.el9.x86_ 10 MB/s | 3.9 MB 00:00 2024-09-11T08:30:43.124 INFO:teuthology.orchestra.run.smithi189.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-11T08:30:43.214 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T08:30:43.215 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:43.215 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:30:43.215 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:43.215 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:43.215 INFO:teuthology.orchestra.run.smithi131.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T08:30:43.215 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T08:30:43.215 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:43.215 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:30:43.299 INFO:teuthology.orchestra.run.smithi189.stdout:(7/34): ceph-osd-17.2.7-1623.gd0e6828a.el9.x86_ 30 MB/s | 15 MB 00:00 2024-09-11T08:30:43.324 INFO:teuthology.orchestra.run.smithi189.stdout:(8/34): python3-ply-3.11-14.el9.noarch.rpm 532 kB/s | 106 kB 00:00 2024-09-11T08:30:43.357 INFO:teuthology.orchestra.run.smithi189.stdout:(9/34): python3-cryptography-36.0.1-4.el9.x86_6 3.2 MB/s | 1.2 MB 00:00 2024-09-11T08:30:43.383 INFO:teuthology.orchestra.run.smithi189.stdout:(10/34): python3-pycparser-2.20-6.el9.noarch.rp 1.6 MB/s | 135 kB 00:00 2024-09-11T08:30:43.425 INFO:teuthology.orchestra.run.smithi189.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-09-11T08:30:43.484 INFO:teuthology.orchestra.run.smithi189.stdout:(12/34): python3-requests-2.25.1-8.el9.noarch.r 784 kB/s | 125 kB 00:00 2024-09-11T08:30:43.509 INFO:teuthology.orchestra.run.smithi189.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 415 kB/s | 35 kB 00:00 2024-09-11T08:30:43.511 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph 2024-09-11T08:30:43.534 INFO:teuthology.orchestra.run.smithi189.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-11T08:30:43.568 INFO:teuthology.orchestra.run.smithi189.stdout:(15/34): python3-pytz-2021.1-5.el9.noarch.rpm 871 kB/s | 51 kB 00:00 2024-09-11T08:30:43.593 INFO:teuthology.orchestra.run.smithi189.stdout:(16/34): python3-toml-0.10.2-6.el9.noarch.rpm 712 kB/s | 42 kB 00:00 2024-09-11T08:30:43.627 INFO:teuthology.orchestra.run.smithi189.stdout:(17/34): python3-packaging-20.9-5.el9.noarch.rp 540 kB/s | 77 kB 00:00 2024-09-11T08:30:43.677 INFO:teuthology.orchestra.run.smithi189.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 398 kB/s | 43 kB 00:00 2024-09-11T08:30:43.702 INFO:teuthology.orchestra.run.smithi189.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 459 kB/s | 11 kB 00:00 2024-09-11T08:30:43.727 INFO:teuthology.orchestra.run.smithi189.stdout:(20/34): python3-jaraco-classes-3.2.1-5.el9.noa 711 kB/s | 18 kB 00:00 2024-09-11T08:30:43.753 INFO:teuthology.orchestra.run.smithi189.stdout:(21/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-11T08:30:43.778 INFO:teuthology.orchestra.run.smithi189.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-11T08:30:43.795 INFO:teuthology.orchestra.run.smithi189.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 464 kB/s | 19 kB 00:00 2024-09-11T08:30:43.820 INFO:teuthology.orchestra.run.smithi189.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 467 kB/s | 20 kB 00:00 2024-09-11T08:30:43.845 INFO:teuthology.orchestra.run.smithi189.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-09-11T08:30:43.871 INFO:teuthology.orchestra.run.smithi189.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-11T08:30:43.896 INFO:teuthology.orchestra.run.smithi189.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-09-11T08:30:43.913 INFO:teuthology.orchestra.run.smithi189.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 391 kB/s | 16 kB 00:00 2024-09-11T08:30:43.938 INFO:teuthology.orchestra.run.smithi189.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-09-11T08:30:43.963 INFO:teuthology.orchestra.run.smithi189.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 712 kB/s | 36 kB 00:00 2024-09-11T08:30:43.989 INFO:teuthology.orchestra.run.smithi189.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-11T08:30:44.006 INFO:teuthology.orchestra.run.smithi189.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-11T08:30:44.031 INFO:teuthology.orchestra.run.smithi189.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-09-11T08:30:44.056 INFO:teuthology.orchestra.run.smithi189.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-09-11T08:30:44.060 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:44.060 INFO:teuthology.orchestra.run.smithi189.stdout:Total 14 MB/s | 27 MB 00:01 2024-09-11T08:30:44.103 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:38 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:30:44.186 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:30:44.224 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:30:44.224 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:30:44.250 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:30:44.253 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:30:44.253 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-11T08:30:44.253 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:30:44.253 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:30:44.253 INFO:teuthology.orchestra.run.smithi131.stdout: ceph x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 6.5 k 2024-09-11T08:30:44.253 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-11T08:30:44.253 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.1 M 2024-09-11T08:30:44.253 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 1.4 M 2024-09-11T08:30:44.254 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 240 k 2024-09-11T08:30:44.254 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.9 M 2024-09-11T08:30:44.254 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 15 M 2024-09-11T08:30:44.254 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T08:30:44.254 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T08:30:44.254 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T08:30:44.254 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T08:30:44.254 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T08:30:44.254 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T08:30:44.254 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T08:30:44.254 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections 2024-09-11T08:30:44.254 INFO:teuthology.orchestra.run.smithi131.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-11T08:30:44.254 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T08:30:44.254 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T08:30:44.255 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T08:30:44.255 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T08:30:44.255 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T08:30:44.255 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T08:30:44.255 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T08:30:44.255 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T08:30:44.255 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T08:30:44.255 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T08:30:44.255 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T08:30:44.255 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T08:30:44.255 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T08:30:44.255 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T08:30:44.255 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T08:30:44.256 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T08:30:44.256 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T08:30:44.256 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T08:30:44.256 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T08:30:44.256 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T08:30:44.256 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:44.256 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:30:44.256 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:30:44.256 INFO:teuthology.orchestra.run.smithi131.stdout:Install 34 Packages 2024-09-11T08:30:44.256 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:44.257 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 27 M 2024-09-11T08:30:44.257 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 96 M 2024-09-11T08:30:44.257 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:30:44.628 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:30:44.628 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:30:45.109 INFO:teuthology.orchestra.run.smithi131.stdout:(1/34): ceph-17.2.7-1623.gd0e6828a.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-11T08:30:45.326 INFO:teuthology.orchestra.run.smithi131.stdout:(2/34): ceph-mds-17.2.7-1623.gd0e6828a.el9.x86_ 6.0 MB/s | 2.1 MB 00:00 2024-09-11T08:30:45.360 INFO:teuthology.orchestra.run.smithi131.stdout:(3/34): ceph-mgr-17.2.7-1623.gd0e6828a.el9.x86_ 3.7 MB/s | 1.4 MB 00:00 2024-09-11T08:30:45.451 INFO:teuthology.orchestra.run.smithi131.stdout:(4/34): ceph-mon-17.2.7-1623.gd0e6828a.el9.x86_ 11 MB/s | 3.9 MB 00:00 2024-09-11T08:30:45.485 INFO:teuthology.orchestra.run.smithi131.stdout:(5/34): ceph-mgr-modules-core-17.2.7-1623.gd0e6 1.9 MB/s | 240 kB 00:00 2024-09-11T08:30:45.627 INFO:teuthology.orchestra.run.smithi131.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-11T08:30:45.677 INFO:teuthology.orchestra.run.smithi131.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-11T08:30:45.831 INFO:teuthology.orchestra.run.smithi131.stdout:(8/34): ceph-osd-17.2.7-1623.gd0e6828a.el9.x86_ 30 MB/s | 15 MB 00:00 2024-09-11T08:30:45.856 INFO:teuthology.orchestra.run.smithi131.stdout:(9/34): python3-pycparser-2.20-6.el9.noarch.rpm 757 kB/s | 135 kB 00:00 2024-09-11T08:30:45.875 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:30:45.890 INFO:teuthology.orchestra.run.smithi131.stdout:(10/34): python3-cryptography-36.0.1-4.el9.x86_ 3.0 MB/s | 1.2 MB 00:00 2024-09-11T08:30:45.926 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-11T08:30:45.940 INFO:teuthology.orchestra.run.smithi131.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-11T08:30:45.958 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-11T08:30:45.991 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-11T08:30:45.999 INFO:teuthology.orchestra.run.smithi131.stdout:(12/34): python3-requests-2.25.1-8.el9.noarch.r 744 kB/s | 125 kB 00:00 2024-09-11T08:30:46.083 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-11T08:30:46.161 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-11T08:30:46.210 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-11T08:30:46.233 INFO:teuthology.orchestra.run.smithi131.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 119 kB/s | 35 kB 00:00 2024-09-11T08:30:46.253 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-11T08:30:46.291 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-11T08:30:46.325 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-11T08:30:46.367 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-11T08:30:46.407 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-11T08:30:46.433 INFO:teuthology.orchestra.run.smithi131.stdout:(14/34): python3-pytz-2021.1-5.el9.noarch.rpm 255 kB/s | 51 kB 00:00 2024-09-11T08:30:46.443 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-11T08:30:46.458 INFO:teuthology.orchestra.run.smithi131.stdout:(15/34): python3-mako-1.1.4-6.el9.noarch.rpm 303 kB/s | 172 kB 00:00 2024-09-11T08:30:46.477 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-11T08:30:46.484 INFO:teuthology.orchestra.run.smithi131.stdout:(16/34): python3-packaging-20.9-5.el9.noarch.rp 159 kB/s | 77 kB 00:00 2024-09-11T08:30:46.526 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-11T08:30:46.526 INFO:teuthology.orchestra.run.smithi131.stdout:(17/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 649 kB/s | 43 kB 00:00 2024-09-11T08:30:46.551 INFO:teuthology.orchestra.run.smithi131.stdout:(18/34): python3-toml-0.10.2-6.el9.noarch.rpm 354 kB/s | 42 kB 00:00 2024-09-11T08:30:46.557 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T08:30:46.584 INFO:teuthology.orchestra.run.smithi131.stdout:(19/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-11T08:30:46.610 INFO:teuthology.orchestra.run.smithi131.stdout:(20/34): python3-jaraco-classes-3.2.1-5.el9.noa 759 kB/s | 18 kB 00:00 2024-09-11T08:30:46.627 INFO:teuthology.orchestra.run.smithi131.stdout:(21/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 141 kB/s | 11 kB 00:00 2024-09-11T08:30:46.631 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-11T08:30:46.652 INFO:teuthology.orchestra.run.smithi131.stdout:(22/34): python3-cherrypy-18.6.1-2.el9.noarch.r 2.8 MB/s | 358 kB 00:00 2024-09-11T08:30:46.669 INFO:teuthology.orchestra.run.smithi131.stdout:(23/34): python3-jaraco-collections-3.0.0-8.el9 394 kB/s | 23 kB 00:00 2024-09-11T08:30:46.686 INFO:teuthology.orchestra.run.smithi131.stdout:(24/34): python3-jaraco-functools-3.5.0-2.el9.n 331 kB/s | 19 kB 00:00 2024-09-11T08:30:46.703 INFO:teuthology.orchestra.run.smithi131.stdout:(25/34): python3-jaraco-text-3.2.0-6.el9.noarch 589 kB/s | 20 kB 00:00 2024-09-11T08:30:46.719 INFO:teuthology.orchestra.run.smithi131.stdout:(26/34): python3-logutils-0.3.5-21.el9.noarch.r 921 kB/s | 46 kB 00:00 2024-09-11T08:30:46.745 INFO:teuthology.orchestra.run.smithi131.stdout:(27/34): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-09-11T08:30:46.770 INFO:teuthology.orchestra.run.smithi131.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-11T08:30:46.778 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-11T08:30:46.795 INFO:teuthology.orchestra.run.smithi131.stdout:(29/34): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-11T08:30:46.821 INFO:teuthology.orchestra.run.smithi131.stdout:(30/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-11T08:30:46.846 INFO:teuthology.orchestra.run.smithi131.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-11T08:30:46.872 INFO:teuthology.orchestra.run.smithi131.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 794 kB/s | 20 kB 00:00 2024-09-11T08:30:46.874 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-11T08:30:46.896 INFO:teuthology.orchestra.run.smithi131.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-11T08:30:46.927 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-11T08:30:46.930 INFO:teuthology.orchestra.run.smithi131.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-11T08:30:46.934 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:46.934 INFO:teuthology.orchestra.run.smithi131.stdout:Total 10 MB/s | 27 MB 00:02 2024-09-11T08:30:46.965 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-11T08:30:47.023 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-11T08:30:47.061 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:30:47.078 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-11T08:30:47.098 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:30:47.098 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:30:47.266 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-11T08:30:47.328 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-11T08:30:47.406 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-11T08:30:47.484 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:30:47.484 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:30:47.511 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-11T08:30:47.563 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-11T08:30:47.685 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-11T08:30:47.839 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 29/34 2024-09-11T08:30:47.870 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T08:30:47.896 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T08:30:47.896 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:30:47.896 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T08:30:47.897 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T08:30:47.897 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T08:30:47.897 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:48.727 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:30:48.850 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-11T08:30:48.882 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-11T08:30:48.915 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-11T08:30:49.006 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-11T08:30:49.076 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-11T08:30:49.126 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-11T08:30:49.169 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-11T08:30:49.199 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-11T08:30:49.213 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T08:30:49.234 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-11T08:30:49.241 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T08:30:49.241 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:30:49.241 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T08:30:49.241 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T08:30:49.241 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T08:30:49.241 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:49.267 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-11T08:30:49.308 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-11T08:30:49.342 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-11T08:30:49.385 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-11T08:30:49.434 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-11T08:30:49.466 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T08:30:49.538 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-11T08:30:49.623 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T08:30:49.651 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T08:30:49.651 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:30:49.651 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T08:30:49.651 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T08:30:49.652 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T08:30:49.652 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:49.676 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-11T08:30:49.762 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-11T08:30:49.815 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-11T08:30:49.857 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T08:30:49.863 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-11T08:30:49.882 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T08:30:49.882 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:30:49.882 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T08:30:49.882 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T08:30:49.882 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T08:30:49.883 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:49.912 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-11T08:30:49.946 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T08:30:49.966 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-11T08:30:50.149 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-11T08:30:50.200 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-11T08:30:50.270 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-11T08:30:50.365 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-11T08:30:50.422 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-11T08:30:50.539 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-11T08:30:50.687 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 29/34 2024-09-11T08:30:50.719 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T08:30:50.744 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T08:30:50.745 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:30:50.745 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T08:30:50.745 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T08:30:50.745 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T08:30:50.745 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T08:30:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/34 2024-09-11T08:30:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/34 2024-09-11T08:30:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/34 2024-09-11T08:30:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/34 2024-09-11T08:30:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/34 2024-09-11T08:30:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 6/34 2024-09-11T08:30:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-11T08:30:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-11T08:30:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-11T08:30:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-11T08:30:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-11T08:30:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-11T08:30:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-11T08:30:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-11T08:30:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T08:30:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-11T08:30:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-11T08:30:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-11T08:30:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-11T08:30:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-11T08:30:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-11T08:30:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-11T08:30:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-11T08:30:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-11T08:30:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-11T08:30:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-11T08:30:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-11T08:30:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-11T08:30:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-11T08:30:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-11T08:30:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-11T08:30:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-11T08:30:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-11T08:30:52.062 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T08:30:52.089 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T08:30:52.089 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:30:52.090 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T08:30:52.090 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T08:30:52.090 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T08:30:52.090 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:52.471 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T08:30:52.495 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T08:30:52.495 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:30:52.495 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T08:30:52.495 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T08:30:52.495 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T08:30:52.495 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:52.705 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T08:30:52.725 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-11T08:30:52.725 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:52.725 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:30:52.725 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:52.725 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:52.725 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:52.725 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:30:52.725 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:52.725 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:52.725 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T08:30:52.725 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T08:30:52.725 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T08:30:52.726 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T08:30:52.726 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T08:30:52.726 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T08:30:52.726 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T08:30:52.726 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T08:30:52.726 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T08:30:52.726 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T08:30:52.726 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T08:30:52.726 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T08:30:52.726 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T08:30:52.726 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T08:30:52.726 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T08:30:52.726 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T08:30:52.726 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T08:30:52.727 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T08:30:52.727 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T08:30:52.727 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T08:30:52.727 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T08:30:52.727 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T08:30:52.727 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T08:30:52.727 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T08:30:52.727 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T08:30:52.727 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T08:30:52.727 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T08:30:52.727 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T08:30:52.727 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:52.727 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:30:52.728 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T08:30:52.729 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:30:52.729 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T08:30:52.729 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T08:30:52.729 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T08:30:52.729 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:52.792 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T08:30:53.072 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-base 2024-09-11T08:30:53.635 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:49 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:30:53.726 INFO:teuthology.orchestra.run.smithi189.stdout:Package ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:53.767 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:30:53.768 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-11T08:30:53.768 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:30:53.840 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install cephadm 2024-09-11T08:30:54.404 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:50 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:30:54.438 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T08:30:54.438 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/34 2024-09-11T08:30:54.438 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/34 2024-09-11T08:30:54.438 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/34 2024-09-11T08:30:54.438 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/34 2024-09-11T08:30:54.438 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/34 2024-09-11T08:30:54.438 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 6/34 2024-09-11T08:30:54.438 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-11T08:30:54.438 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-11T08:30:54.438 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-11T08:30:54.438 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-11T08:30:54.439 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-11T08:30:54.439 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-11T08:30:54.439 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-11T08:30:54.439 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-11T08:30:54.439 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T08:30:54.439 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-11T08:30:54.439 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-11T08:30:54.439 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-11T08:30:54.439 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-11T08:30:54.439 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-11T08:30:54.439 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-11T08:30:54.439 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-11T08:30:54.439 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-11T08:30:54.439 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-11T08:30:54.440 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-11T08:30:54.440 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-11T08:30:54.440 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-11T08:30:54.440 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-11T08:30:54.440 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-11T08:30:54.440 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-11T08:30:54.440 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-11T08:30:54.440 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-11T08:30:54.440 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-11T08:30:54.532 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:30:54.533 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:30:54.533 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-11T08:30:54.533 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:30:54.533 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:30:54.533 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 83 k 2024-09-11T08:30:54.533 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:54.533 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:30:54.534 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:30:54.534 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-11T08:30:54.534 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:54.534 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 83 k 2024-09-11T08:30:54.534 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 375 k 2024-09-11T08:30:54.534 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:30:54.732 INFO:teuthology.orchestra.run.smithi189.stdout:cephadm-17.2.7-1623.gd0e6828a.el9.noarch.rpm 419 kB/s | 83 kB 00:00 2024-09-11T08:30:54.732 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:54.732 INFO:teuthology.orchestra.run.smithi189.stdout:Total 416 kB/s | 83 kB 00:00 2024-09-11T08:30:54.733 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:30:54.738 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:30:54.738 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:30:54.740 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:30:54.740 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:30:54.818 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:30:55.231 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:30:55.317 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:30:55.506 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-11T08:30:55.506 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:55.506 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:30:55.507 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:55.507 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:55.507 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:55.507 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:30:55.507 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:55.507 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:55.507 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T08:30:55.507 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T08:30:55.507 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T08:30:55.507 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T08:30:55.507 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T08:30:55.507 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T08:30:55.508 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T08:30:55.508 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T08:30:55.508 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T08:30:55.508 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T08:30:55.508 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T08:30:55.508 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T08:30:55.508 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T08:30:55.508 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T08:30:55.508 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T08:30:55.508 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T08:30:55.508 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T08:30:55.508 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T08:30:55.508 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T08:30:55.508 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T08:30:55.509 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T08:30:55.509 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T08:30:55.509 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T08:30:55.509 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T08:30:55.509 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T08:30:55.509 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T08:30:55.509 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T08:30:55.509 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T08:30:55.509 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:55.509 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:30:55.661 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:30:55.910 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-base 2024-09-11T08:30:55.925 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:30:55.925 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:55.925 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:30:55.925 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:30:55.925 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:55.925 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:30:56.151 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T08:30:56.481 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:50 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:30:56.572 INFO:teuthology.orchestra.run.smithi131.stdout:Package ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:56.613 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:30:56.614 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-11T08:30:56.614 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:30:56.681 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install cephadm 2024-09-11T08:30:56.724 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:52 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:30:56.860 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:30:56.861 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:30:56.861 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2024-09-11T08:30:56.861 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:30:56.862 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:30:56.862 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 137 k 2024-09-11T08:30:56.862 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:56.862 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:30:56.862 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:30:56.862 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-11T08:30:56.862 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:56.862 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 137 k 2024-09-11T08:30:56.863 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 379 k 2024-09-11T08:30:56.863 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:30:57.062 INFO:teuthology.orchestra.run.smithi189.stdout:ceph-immutable-object-cache-17.2.7-1623.gd0e682 686 kB/s | 137 kB 00:00 2024-09-11T08:30:57.063 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:57.063 INFO:teuthology.orchestra.run.smithi189.stdout:Total 681 kB/s | 137 kB 00:00 2024-09-11T08:30:57.063 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:30:57.070 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:30:57.071 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:30:57.129 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:30:57.129 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:30:57.236 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:51 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:30:57.290 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:30:57.363 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:30:57.366 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:30:57.367 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:30:57.367 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-11T08:30:57.367 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:30:57.367 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:30:57.367 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 83 k 2024-09-11T08:30:57.367 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:57.367 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:30:57.367 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:30:57.367 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-11T08:30:57.367 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:57.368 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 83 k 2024-09-11T08:30:57.368 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 375 k 2024-09-11T08:30:57.368 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:30:57.389 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:30:57.389 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:30:57.389 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T08:30:57.389 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:57.538 INFO:teuthology.orchestra.run.smithi131.stdout:cephadm-17.2.7-1623.gd0e6828a.el9.noarch.rpm 486 kB/s | 83 kB 00:00 2024-09-11T08:30:57.539 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:57.539 INFO:teuthology.orchestra.run.smithi131.stdout:Total 482 kB/s | 83 kB 00:00 2024-09-11T08:30:57.539 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:30:57.544 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:30:57.544 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:30:57.546 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:30:57.546 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:30:57.625 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:30:58.120 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:30:58.200 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:30:58.306 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:30:58.306 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:58.306 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:30:58.307 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:30:58.307 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:30:58.307 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:30:58.514 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr 2024-09-11T08:30:58.544 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:30:58.810 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:30:58.811 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:58.811 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:30:58.811 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:30:58.811 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:58.811 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:30:59.061 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T08:30:59.089 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:55 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:30:59.181 INFO:teuthology.orchestra.run.smithi189.stdout:Package ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:30:59.222 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:30:59.223 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-11T08:30:59.224 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:30:59.298 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T08:30:59.642 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:53 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:30:59.780 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:30:59.781 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:30:59.781 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repo Size 2024-09-11T08:30:59.781 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:30:59.781 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:30:59.781 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 137 k 2024-09-11T08:30:59.782 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:59.782 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:30:59.782 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:30:59.782 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-11T08:30:59.782 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:30:59.782 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 137 k 2024-09-11T08:30:59.782 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 379 k 2024-09-11T08:30:59.782 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:30:59.863 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:55 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:30:59.970 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-immutable-object-cache-17.2.7-1623.gd0e682 731 kB/s | 137 kB 00:00 2024-09-11T08:30:59.970 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:30:59.971 INFO:teuthology.orchestra.run.smithi131.stdout:Total 726 kB/s | 137 kB 00:00 2024-09-11T08:30:59.971 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:30:59.978 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:30:59.978 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:31:00.007 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:00.008 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:00.008 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-11T08:31:00.008 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:00.008 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:31:00.008 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 3.4 M 2024-09-11T08:31:00.008 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-11T08:31:00.009 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 24 k 2024-09-11T08:31:00.009 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 15 k 2024-09-11T08:31:00.009 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-11T08:31:00.009 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T08:31:00.009 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T08:31:00.009 INFO:teuthology.orchestra.run.smithi189.stdout:Installing weak dependencies: 2024-09-11T08:31:00.009 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-11T08:31:00.009 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:00.009 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:31:00.009 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:00.009 INFO:teuthology.orchestra.run.smithi189.stdout:Install 7 Packages 2024-09-11T08:31:00.009 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:00.010 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 3.7 M 2024-09-11T08:31:00.010 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 80 M 2024-09-11T08:31:00.010 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:31:00.036 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:31:00.036 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:31:00.191 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:31:00.262 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:31:00.289 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:31:00.289 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:31:00.289 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T08:31:00.289 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:00.315 INFO:teuthology.orchestra.run.smithi189.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1623.gd0e 164 kB/s | 24 kB 00:00 2024-09-11T08:31:00.341 INFO:teuthology.orchestra.run.smithi189.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1623.gd0e6 84 kB/s | 15 kB 00:00 2024-09-11T08:31:00.383 INFO:teuthology.orchestra.run.smithi189.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 136 kB/s | 9.0 kB 00:00 2024-09-11T08:31:00.408 INFO:teuthology.orchestra.run.smithi189.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-11T08:31:00.433 INFO:teuthology.orchestra.run.smithi189.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 445 kB/s | 41 kB 00:00 2024-09-11T08:31:00.458 INFO:teuthology.orchestra.run.smithi189.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-11T08:31:00.600 INFO:teuthology.orchestra.run.smithi189.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1623.gd0e6828a 7.7 MB/s | 3.4 MB 00:00 2024-09-11T08:31:00.604 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:00.604 INFO:teuthology.orchestra.run.smithi189.stdout:Total 6.1 MB/s | 3.7 MB 00:00 2024-09-11T08:31:00.604 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:31:00.612 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:31:00.612 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:31:00.714 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:31:00.714 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:31:01.023 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:31:01.093 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-11T08:31:01.127 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-11T08:31:01.175 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-11T08:31:01.206 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-11T08:31:01.232 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T08:31:01.232 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:01.232 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:31:01.232 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:31:01.232 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:01.233 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:01.234 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 5/7 2024-09-11T08:31:01.441 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr 2024-09-11T08:31:02.006 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:56 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:02.098 INFO:teuthology.orchestra.run.smithi131.stdout:Package ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:31:02.139 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:02.141 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-11T08:31:02.141 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:02.210 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T08:31:02.229 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 6/7 2024-09-11T08:31:02.271 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T08:31:02.768 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:56 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:02.902 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:02.902 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:02.902 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-11T08:31:02.902 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:02.902 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:31:02.903 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 3.4 M 2024-09-11T08:31:02.903 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-11T08:31:02.903 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 24 k 2024-09-11T08:31:02.903 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 15 k 2024-09-11T08:31:02.903 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-11T08:31:02.903 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T08:31:02.903 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T08:31:02.903 INFO:teuthology.orchestra.run.smithi131.stdout:Installing weak dependencies: 2024-09-11T08:31:02.903 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-11T08:31:02.903 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:02.903 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:31:02.903 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:02.903 INFO:teuthology.orchestra.run.smithi131.stdout:Install 7 Packages 2024-09-11T08:31:02.903 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:02.904 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 3.7 M 2024-09-11T08:31:02.904 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 80 M 2024-09-11T08:31:02.904 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:31:03.215 INFO:teuthology.orchestra.run.smithi131.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1623.gd0e6 91 kB/s | 15 kB 00:00 2024-09-11T08:31:03.241 INFO:teuthology.orchestra.run.smithi131.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1623.gd0e 132 kB/s | 24 kB 00:00 2024-09-11T08:31:03.282 INFO:teuthology.orchestra.run.smithi131.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-11T08:31:03.308 INFO:teuthology.orchestra.run.smithi131.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-11T08:31:03.333 INFO:teuthology.orchestra.run.smithi131.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 445 kB/s | 41 kB 00:00 2024-09-11T08:31:03.358 INFO:teuthology.orchestra.run.smithi131.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-11T08:31:03.385 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T08:31:03.385 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 1/7 2024-09-11T08:31:03.386 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 2/7 2024-09-11T08:31:03.386 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 3/7 2024-09-11T08:31:03.386 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-11T08:31:03.386 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-11T08:31:03.386 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-11T08:31:03.434 INFO:teuthology.orchestra.run.smithi131.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1623.gd0e6828a 8.9 MB/s | 3.4 MB 00:00 2024-09-11T08:31:03.437 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:03.437 INFO:teuthology.orchestra.run.smithi131.stdout:Total 6.8 MB/s | 3.7 MB 00:00 2024-09-11T08:31:03.437 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:31:03.445 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:31:03.445 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:31:03.531 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:31:03.531 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:31:03.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-11T08:31:03.658 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:03.658 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:31:03.658 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:31:03.658 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:31:03.658 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:31:03.659 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-11T08:31:03.659 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-11T08:31:03.659 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T08:31:03.659 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T08:31:03.659 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:03.659 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:03.837 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:31:03.899 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T08:31:03.902 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-11T08:31:03.936 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-11T08:31:03.982 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-11T08:31:04.023 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-11T08:31:04.051 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 5/7 2024-09-11T08:31:04.462 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:00 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:04.597 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:04.598 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================= 2024-09-11T08:31:04.598 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-11T08:31:04.598 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================= 2024-09-11T08:31:04.598 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:31:04.598 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 7.4 M 2024-09-11T08:31:04.598 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-11T08:31:04.598 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T08:31:04.599 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T08:31:04.599 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T08:31:04.599 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T08:31:04.599 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T08:31:04.599 INFO:teuthology.orchestra.run.smithi189.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T08:31:04.599 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T08:31:04.599 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T08:31:04.599 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T08:31:04.599 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T08:31:04.599 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T08:31:04.599 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:04.599 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:31:04.599 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================= 2024-09-11T08:31:04.599 INFO:teuthology.orchestra.run.smithi189.stdout:Install 12 Packages 2024-09-11T08:31:04.600 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:04.600 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 38 M 2024-09-11T08:31:04.600 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 212 M 2024-09-11T08:31:04.600 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:31:04.974 INFO:teuthology.orchestra.run.smithi189.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-11T08:31:05.033 INFO:teuthology.orchestra.run.smithi189.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.2 MB/s | 801 kB 00:00 2024-09-11T08:31:05.046 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 6/7 2024-09-11T08:31:05.080 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T08:31:05.300 INFO:teuthology.orchestra.run.smithi189.stdout:(3/12): ceph-mgr-diskprediction-local-17.2.7-16 16 MB/s | 7.4 MB 00:00 2024-09-11T08:31:05.325 INFO:teuthology.orchestra.run.smithi189.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 93 kB/s | 33 kB 00:00 2024-09-11T08:31:05.525 INFO:teuthology.orchestra.run.smithi189.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 192 kB/s | 38 kB 00:00 2024-09-11T08:31:05.576 INFO:teuthology.orchestra.run.smithi189.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 66 kB/s | 18 kB 00:00 2024-09-11T08:31:06.043 INFO:teuthology.orchestra.run.smithi189.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 524 kB/s | 245 kB 00:00 2024-09-11T08:31:06.101 INFO:teuthology.orchestra.run.smithi189.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.8 MB/s | 3.0 MB 00:01 2024-09-11T08:31:06.203 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T08:31:06.203 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 1/7 2024-09-11T08:31:06.203 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 2/7 2024-09-11T08:31:06.203 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 3/7 2024-09-11T08:31:06.203 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-11T08:31:06.203 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-11T08:31:06.203 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-11T08:31:06.218 INFO:teuthology.orchestra.run.smithi189.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-09-11T08:31:06.481 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-11T08:31:06.481 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:06.481 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:31:06.481 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:31:06.482 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:31:06.482 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:31:06.482 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-11T08:31:06.482 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-11T08:31:06.482 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T08:31:06.482 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T08:31:06.482 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:06.482 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:06.544 INFO:teuthology.orchestra.run.smithi189.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.9 MB/s | 4.9 MB 00:01 2024-09-11T08:31:06.718 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T08:31:06.793 INFO:teuthology.orchestra.run.smithi189.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.8 MB/s | 5.1 MB 00:00 2024-09-11T08:31:07.276 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:01 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:07.410 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:07.410 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-09-11T08:31:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-11T08:31:07.411 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-09-11T08:31:07.411 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:31:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 7.4 M 2024-09-11T08:31:07.411 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-11T08:31:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T08:31:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T08:31:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T08:31:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T08:31:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T08:31:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T08:31:07.411 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T08:31:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T08:31:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T08:31:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T08:31:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T08:31:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:07.412 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:31:07.412 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-09-11T08:31:07.412 INFO:teuthology.orchestra.run.smithi131.stdout:Install 12 Packages 2024-09-11T08:31:07.412 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:07.413 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 38 M 2024-09-11T08:31:07.413 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 212 M 2024-09-11T08:31:07.413 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:31:08.183 INFO:teuthology.orchestra.run.smithi131.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 883 kB/s | 191 kB 00:00 2024-09-11T08:31:08.275 INFO:teuthology.orchestra.run.smithi131.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-11T08:31:08.459 INFO:teuthology.orchestra.run.smithi131.stdout:(3/12): ceph-mgr-diskprediction-local-17.2.7-16 15 MB/s | 7.4 MB 00:00 2024-09-11T08:31:08.484 INFO:teuthology.orchestra.run.smithi131.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 109 kB/s | 33 kB 00:00 2024-09-11T08:31:08.693 INFO:teuthology.orchestra.run.smithi131.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 185 kB/s | 38 kB 00:00 2024-09-11T08:31:08.743 INFO:teuthology.orchestra.run.smithi131.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 64 kB/s | 18 kB 00:00 2024-09-11T08:31:09.202 INFO:teuthology.orchestra.run.smithi131.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 534 kB/s | 245 kB 00:00 2024-09-11T08:31:10.085 INFO:teuthology.orchestra.run.smithi131.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 1.7 MB/s | 3.0 MB 00:01 2024-09-11T08:31:10.211 INFO:teuthology.orchestra.run.smithi131.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 250 kB 00:00 2024-09-11T08:31:11.244 INFO:teuthology.orchestra.run.smithi131.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 1.9 MB/s | 4.9 MB 00:02 2024-09-11T08:31:12.011 INFO:teuthology.orchestra.run.smithi131.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.8 MB/s | 5.1 MB 00:02 2024-09-11T08:31:15.269 INFO:teuthology.orchestra.run.smithi131.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.2 MB/s | 16 MB 00:05 2024-09-11T08:31:15.270 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:15.270 INFO:teuthology.orchestra.run.smithi131.stdout:Total 4.9 MB/s | 38 MB 00:07 2024-09-11T08:31:15.623 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:31:15.648 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:31:15.648 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:31:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:31:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:31:16.944 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:31:17.031 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T08:31:17.075 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:31:17.172 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:31:17.235 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T08:31:17.420 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T08:31:17.450 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T08:31:17.535 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T08:31:18.161 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T08:31:18.276 INFO:teuthology.orchestra.run.smithi189.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.3 MB/s | 16 MB 00:12 2024-09-11T08:31:18.277 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:18.277 INFO:teuthology.orchestra.run.smithi189.stdout:Total 2.8 MB/s | 38 MB 00:13 2024-09-11T08:31:18.279 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T08:31:18.637 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:31:18.664 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:31:18.664 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:31:19.214 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:31:19.214 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:31:19.508 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:31:19.973 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:31:20.054 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T08:31:20.098 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:31:20.202 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:31:20.250 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T08:31:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T08:31:20.465 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T08:31:20.552 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T08:31:20.683 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T08:31:20.717 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T08:31:21.193 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T08:31:21.312 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T08:31:22.571 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:31:23.625 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T08:31:23.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 1/12 2024-09-11T08:31:23.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:31:23.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:31:23.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T08:31:23.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T08:31:23.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T08:31:23.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T08:31:23.625 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T08:31:23.626 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T08:31:23.626 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:31:23.626 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T08:31:23.632 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T08:31:23.673 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T08:31:24.028 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T08:31:24.028 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:24.028 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:31:24.028 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:31:24.028 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T08:31:24.029 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T08:31:24.029 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T08:31:24.029 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T08:31:24.029 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T08:31:24.029 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T08:31:24.029 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T08:31:24.029 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T08:31:24.029 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T08:31:24.029 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T08:31:24.029 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T08:31:24.029 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:24.029 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:24.419 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr-rook 2024-09-11T08:31:24.996 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:18 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:25.132 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:25.133 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:25.133 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-11T08:31:25.133 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:25.133 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:31:25.133 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 47 k 2024-09-11T08:31:25.133 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-11T08:31:25.133 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T08:31:25.134 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T08:31:25.134 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T08:31:25.134 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T08:31:25.134 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T08:31:25.134 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T08:31:25.134 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T08:31:25.134 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T08:31:25.134 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T08:31:25.134 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T08:31:25.134 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T08:31:25.134 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T08:31:25.134 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:25.134 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:31:25.135 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:25.135 INFO:teuthology.orchestra.run.smithi131.stdout:Install 13 Packages 2024-09-11T08:31:25.135 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:25.135 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 2.2 M 2024-09-11T08:31:25.135 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 26 M 2024-09-11T08:31:25.135 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:31:25.428 INFO:teuthology.orchestra.run.smithi131.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 283 kB/s | 26 kB 00:00 2024-09-11T08:31:25.453 INFO:teuthology.orchestra.run.smithi131.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 166 kB/s | 19 kB 00:00 2024-09-11T08:31:25.528 INFO:teuthology.orchestra.run.smithi131.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-11T08:31:25.553 INFO:teuthology.orchestra.run.smithi131.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 222 kB 00:00 2024-09-11T08:31:25.579 INFO:teuthology.orchestra.run.smithi131.stdout:(5/13): ceph-mgr-rook-17.2.7-1623.gd0e6828a.el9 196 kB/s | 47 kB 00:00 2024-09-11T08:31:25.604 INFO:teuthology.orchestra.run.smithi131.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-11T08:31:25.662 INFO:teuthology.orchestra.run.smithi131.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.0 MB/s | 279 kB 00:00 2024-09-11T08:31:25.722 INFO:teuthology.orchestra.run.smithi131.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 226 kB/s | 32 kB 00:00 2024-09-11T08:31:25.747 INFO:teuthology.orchestra.run.smithi131.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 99 kB/s | 14 kB 00:00 2024-09-11T08:31:25.772 INFO:teuthology.orchestra.run.smithi131.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-11T08:31:25.798 INFO:teuthology.orchestra.run.smithi131.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.8 MB/s | 220 kB 00:00 2024-09-11T08:31:25.822 INFO:teuthology.orchestra.run.smithi131.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-11T08:31:25.856 INFO:teuthology.orchestra.run.smithi131.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.7 MB/s | 1.0 MB 00:00 2024-09-11T08:31:25.859 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:25.859 INFO:teuthology.orchestra.run.smithi131.stdout:Total 3.1 MB/s | 2.2 MB 00:00 2024-09-11T08:31:25.923 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:31:25.937 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:31:25.938 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:31:26.028 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:31:26.029 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:31:26.502 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T08:31:26.503 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 1/12 2024-09-11T08:31:26.503 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T08:31:26.503 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T08:31:26.503 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T08:31:26.503 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T08:31:26.503 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T08:31:26.503 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T08:31:26.503 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T08:31:26.503 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T08:31:26.503 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T08:31:26.503 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T08:31:26.529 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:31:26.703 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T08:31:26.757 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T08:31:26.830 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T08:31:26.867 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T08:31:26.908 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T08:31:26.963 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T08:31:26.963 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:26.963 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:31:26.963 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:31:26.963 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T08:31:26.964 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T08:31:26.964 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T08:31:26.964 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T08:31:26.964 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T08:31:26.964 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T08:31:26.964 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T08:31:26.964 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T08:31:26.964 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T08:31:26.964 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T08:31:26.964 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T08:31:26.964 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:26.964 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:27.019 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T08:31:27.121 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T08:31:27.194 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T08:31:27.393 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr-rook 2024-09-11T08:31:27.911 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T08:31:27.958 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T08:31:27.967 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:23 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:27.991 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T08:31:28.037 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T08:31:28.061 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T08:31:28.100 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:28.101 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:28.101 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-11T08:31:28.101 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:28.101 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:31:28.101 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 47 k 2024-09-11T08:31:28.101 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-11T08:31:28.101 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T08:31:28.101 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T08:31:28.102 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T08:31:28.102 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T08:31:28.102 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T08:31:28.102 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T08:31:28.102 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T08:31:28.102 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T08:31:28.102 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T08:31:28.102 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T08:31:28.102 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T08:31:28.102 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T08:31:28.102 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:28.102 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:31:28.103 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:28.103 INFO:teuthology.orchestra.run.smithi189.stdout:Install 13 Packages 2024-09-11T08:31:28.103 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:28.103 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 2.2 M 2024-09-11T08:31:28.104 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 26 M 2024-09-11T08:31:28.104 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:31:28.655 INFO:teuthology.orchestra.run.smithi189.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 220 kB/s | 19 kB 00:00 2024-09-11T08:31:28.681 INFO:teuthology.orchestra.run.smithi189.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 228 kB/s | 26 kB 00:00 2024-09-11T08:31:28.739 INFO:teuthology.orchestra.run.smithi189.stdout:(3/13): ceph-mgr-rook-17.2.7-1623.gd0e6828a.el9 278 kB/s | 47 kB 00:00 2024-09-11T08:31:28.764 INFO:teuthology.orchestra.run.smithi189.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-11T08:31:28.790 INFO:teuthology.orchestra.run.smithi189.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-11T08:31:28.815 INFO:teuthology.orchestra.run.smithi189.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-11T08:31:28.825 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T08:31:28.825 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 1/13 2024-09-11T08:31:28.825 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T08:31:28.825 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T08:31:28.825 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T08:31:28.825 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T08:31:28.825 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T08:31:28.825 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T08:31:28.825 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T08:31:28.825 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T08:31:28.825 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T08:31:28.825 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T08:31:28.825 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T08:31:28.857 INFO:teuthology.orchestra.run.smithi189.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 480 kB/s | 32 kB 00:00 2024-09-11T08:31:28.882 INFO:teuthology.orchestra.run.smithi189.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 211 kB/s | 14 kB 00:00 2024-09-11T08:31:28.907 INFO:teuthology.orchestra.run.smithi189.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.6 MB/s | 279 kB 00:00 2024-09-11T08:31:28.941 INFO:teuthology.orchestra.run.smithi189.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-11T08:31:28.966 INFO:teuthology.orchestra.run.smithi189.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.8 MB/s | 90 kB 00:00 2024-09-11T08:31:28.992 INFO:teuthology.orchestra.run.smithi189.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 705 kB/s | 59 kB 00:00 2024-09-11T08:31:29.025 INFO:teuthology.orchestra.run.smithi189.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-11T08:31:29.028 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:29.028 INFO:teuthology.orchestra.run.smithi189.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-09-11T08:31:29.091 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:31:29.105 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:31:29.106 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:31:29.162 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T08:31:29.162 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:29.162 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:31:29.162 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:31:29.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T08:31:29.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T08:31:29.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T08:31:29.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T08:31:29.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T08:31:29.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T08:31:29.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T08:31:29.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T08:31:29.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T08:31:29.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T08:31:29.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T08:31:29.163 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T08:31:29.163 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:29.163 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:29.193 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:31:29.196 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:31:29.441 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T08:31:29.745 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:31:29.914 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T08:31:29.976 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T08:31:30.000 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:24 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:30.032 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T08:31:30.069 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T08:31:30.110 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T08:31:30.135 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:30.136 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:30.136 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-11T08:31:30.136 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:30.136 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:31:30.136 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 124 k 2024-09-11T08:31:30.136 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-11T08:31:30.136 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T08:31:30.136 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T08:31:30.136 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T08:31:30.136 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T08:31:30.137 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T08:31:30.137 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:30.137 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:31:30.137 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:30.137 INFO:teuthology.orchestra.run.smithi131.stdout:Install 6 Packages 2024-09-11T08:31:30.137 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:30.137 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 7.0 M 2024-09-11T08:31:30.138 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 33 M 2024-09-11T08:31:30.138 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:31:30.212 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T08:31:30.313 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T08:31:30.361 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T08:31:30.917 INFO:teuthology.orchestra.run.smithi131.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.9 MB/s | 248 kB 00:00 2024-09-11T08:31:30.967 INFO:teuthology.orchestra.run.smithi131.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-11T08:31:30.993 INFO:teuthology.orchestra.run.smithi131.stdout:(3/6): ceph-mgr-cephadm-17.2.7-1623.gd0e6828a.e 603 kB/s | 124 kB 00:00 2024-09-11T08:31:31.018 INFO:teuthology.orchestra.run.smithi131.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-11T08:31:31.060 INFO:teuthology.orchestra.run.smithi131.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-11T08:31:31.077 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T08:31:31.126 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T08:31:31.152 INFO:teuthology.orchestra.run.smithi131.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-11T08:31:31.155 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:31.155 INFO:teuthology.orchestra.run.smithi131.stdout:Total 6.9 MB/s | 7.0 MB 00:01 2024-09-11T08:31:31.159 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T08:31:31.205 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T08:31:31.227 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:31:31.229 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T08:31:31.239 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:31:31.239 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:31:31.338 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:31:31.339 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:31:31.650 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:31:31.818 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T08:31:31.871 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T08:31:32.007 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T08:31:32.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 1/13 2024-09-11T08:31:32.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T08:31:32.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T08:31:32.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T08:31:32.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T08:31:32.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T08:31:32.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T08:31:32.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T08:31:32.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T08:31:32.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T08:31:32.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T08:31:32.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T08:31:32.377 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T08:31:32.377 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:32.377 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:31:32.378 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:31:32.378 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T08:31:32.378 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T08:31:32.378 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T08:31:32.378 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T08:31:32.378 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T08:31:32.378 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T08:31:32.378 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T08:31:32.378 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T08:31:32.378 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T08:31:32.378 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T08:31:32.379 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T08:31:32.379 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T08:31:32.379 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:32.379 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:32.386 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T08:31:32.473 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T08:31:32.528 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T08:31:32.558 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T08:31:32.642 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T08:31:33.208 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:29 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:33.342 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:33.343 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:33.343 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-11T08:31:33.343 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:33.343 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:31:33.343 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 124 k 2024-09-11T08:31:33.343 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-11T08:31:33.343 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T08:31:33.343 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T08:31:33.343 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T08:31:33.344 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T08:31:33.344 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T08:31:33.344 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:33.344 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:31:33.344 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:33.344 INFO:teuthology.orchestra.run.smithi189.stdout:Install 6 Packages 2024-09-11T08:31:33.344 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:33.345 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 7.0 M 2024-09-11T08:31:33.345 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 33 M 2024-09-11T08:31:33.345 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:31:33.375 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T08:31:33.375 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/6 2024-09-11T08:31:33.375 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T08:31:33.375 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T08:31:33.375 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T08:31:33.376 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T08:31:33.651 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T08:31:33.652 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:33.652 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:31:33.652 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:31:33.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T08:31:33.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T08:31:33.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T08:31:33.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T08:31:33.652 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T08:31:33.652 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:33.652 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:33.925 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-fuse 2024-09-11T08:31:34.151 INFO:teuthology.orchestra.run.smithi189.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.5 MB/s | 248 kB 00:00 2024-09-11T08:31:34.201 INFO:teuthology.orchestra.run.smithi189.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-11T08:31:34.226 INFO:teuthology.orchestra.run.smithi189.stdout:(3/6): ceph-mgr-cephadm-17.2.7-1623.gd0e6828a.e 527 kB/s | 124 kB 00:00 2024-09-11T08:31:34.252 INFO:teuthology.orchestra.run.smithi189.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-11T08:31:34.293 INFO:teuthology.orchestra.run.smithi189.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-11T08:31:34.360 INFO:teuthology.orchestra.run.smithi189.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-11T08:31:34.363 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:34.363 INFO:teuthology.orchestra.run.smithi189.stdout:Total 6.9 MB/s | 7.0 MB 00:01 2024-09-11T08:31:34.435 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:31:34.446 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:31:34.446 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:31:34.488 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:28 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:34.550 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:31:34.551 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:31:34.622 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:34.622 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:34.622 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-11T08:31:34.622 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:34.622 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:31:34.622 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 813 k 2024-09-11T08:31:34.623 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-11T08:31:34.623 INFO:teuthology.orchestra.run.smithi131.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T08:31:34.623 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:34.623 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:31:34.623 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:34.623 INFO:teuthology.orchestra.run.smithi131.stdout:Install 2 Packages 2024-09-11T08:31:34.623 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:34.623 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 893 k 2024-09-11T08:31:34.624 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 2.6 M 2024-09-11T08:31:34.624 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:31:34.820 INFO:teuthology.orchestra.run.smithi131.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 530 kB/s | 80 kB 00:00 2024-09-11T08:31:34.856 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:31:34.971 INFO:teuthology.orchestra.run.smithi131.stdout:(2/2): ceph-fuse-17.2.7-1623.gd0e6828a.el9.x86_ 2.6 MB/s | 813 kB 00:00 2024-09-11T08:31:34.971 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:34.972 INFO:teuthology.orchestra.run.smithi131.stdout:Total 2.5 MB/s | 893 kB 00:00 2024-09-11T08:31:34.983 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:31:35.004 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:31:35.004 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:31:35.033 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T08:31:35.087 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T08:31:35.088 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:31:35.088 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:31:35.279 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:31:35.403 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T08:31:35.469 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T08:31:35.602 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T08:31:35.689 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T08:31:35.744 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T08:31:35.774 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T08:31:36.115 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T08:31:36.115 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/2 2024-09-11T08:31:36.388 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T08:31:36.389 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:36.389 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:31:36.389 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T08:31:36.389 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:36.389 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:36.539 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T08:31:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/6 2024-09-11T08:31:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T08:31:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T08:31:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T08:31:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T08:31:36.670 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-volume 2024-09-11T08:31:36.801 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T08:31:36.801 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:36.801 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:31:36.801 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:31:36.801 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T08:31:36.802 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T08:31:36.802 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T08:31:36.802 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T08:31:36.802 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T08:31:36.802 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:36.802 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:37.073 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-fuse 2024-09-11T08:31:37.250 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:31 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:37.385 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:37.386 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:37.386 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-11T08:31:37.386 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:37.386 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:31:37.386 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 270 k 2024-09-11T08:31:37.386 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:37.386 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:31:37.386 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:37.386 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-11T08:31:37.386 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:37.387 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 270 k 2024-09-11T08:31:37.387 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 1.3 M 2024-09-11T08:31:37.387 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:31:37.609 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-volume-17.2.7-1623.gd0e6828a.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-11T08:31:37.610 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:37.610 INFO:teuthology.orchestra.run.smithi131.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-11T08:31:37.610 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:31:37.618 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:31:37.618 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:31:37.637 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:33 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:37.649 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:31:37.649 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:31:37.771 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:37.772 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:37.772 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-11T08:31:37.772 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:37.772 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:31:37.773 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 813 k 2024-09-11T08:31:37.773 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-11T08:31:37.773 INFO:teuthology.orchestra.run.smithi189.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T08:31:37.773 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:37.773 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:31:37.773 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:37.773 INFO:teuthology.orchestra.run.smithi189.stdout:Install 2 Packages 2024-09-11T08:31:37.774 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:37.774 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 893 k 2024-09-11T08:31:37.774 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 2.6 M 2024-09-11T08:31:37.774 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:31:37.869 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:31:37.928 INFO:teuthology.orchestra.run.smithi189.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 594 kB/s | 80 kB 00:00 2024-09-11T08:31:37.936 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:31:37.952 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:31:37.953 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:31:37.953 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T08:31:37.953 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:38.328 INFO:teuthology.orchestra.run.smithi189.stdout:(2/2): ceph-fuse-17.2.7-1623.gd0e6828a.el9.x86_ 1.5 MB/s | 813 kB 00:00 2024-09-11T08:31:38.329 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:38.329 INFO:teuthology.orchestra.run.smithi189.stdout:Total 1.6 MB/s | 893 kB 00:00 2024-09-11T08:31:38.341 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:31:38.364 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:31:38.364 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:31:38.448 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:31:38.448 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:31:38.647 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:31:38.793 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T08:31:38.860 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T08:31:38.939 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:31:38.939 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:38.940 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:31:38.940 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:31:38.940 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:38.940 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:39.215 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install librados-devel 2024-09-11T08:31:39.599 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T08:31:39.600 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/2 2024-09-11T08:31:39.793 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:33 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:39.877 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T08:31:39.877 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:39.877 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:31:39.877 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T08:31:39.878 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:39.878 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:39.930 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:39.931 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:39.931 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repo Size 2024-09-11T08:31:39.931 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:39.931 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:31:39.931 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 126 k 2024-09-11T08:31:39.931 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:39.931 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:31:39.931 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:39.931 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-11T08:31:39.931 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:39.931 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 126 k 2024-09-11T08:31:39.931 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 448 k 2024-09-11T08:31:39.931 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:31:40.194 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-volume 2024-09-11T08:31:40.290 INFO:teuthology.orchestra.run.smithi131.stdout:librados-devel-17.2.7-1623.gd0e6828a.el9.x86_64 353 kB/s | 126 kB 00:00 2024-09-11T08:31:40.291 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:40.291 INFO:teuthology.orchestra.run.smithi131.stdout:Total 351 kB/s | 126 kB 00:00 2024-09-11T08:31:40.292 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:31:40.297 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:31:40.297 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:31:40.360 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:31:40.360 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:31:40.518 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:31:40.630 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:40.776 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:40.911 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:40.912 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:40.912 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-11T08:31:40.912 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:40.912 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:31:40.912 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 270 k 2024-09-11T08:31:40.912 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:40.912 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:31:40.912 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:40.913 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-11T08:31:40.913 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:40.913 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 270 k 2024-09-11T08:31:40.913 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 1.3 M 2024-09-11T08:31:40.913 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:31:41.071 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:41.331 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:41.331 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:41.331 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:31:41.332 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:31:41.332 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:41.332 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:41.341 INFO:teuthology.orchestra.run.smithi189.stdout:ceph-volume-17.2.7-1623.gd0e6828a.el9.noarch.rp 631 kB/s | 270 kB 00:00 2024-09-11T08:31:41.341 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:41.342 INFO:teuthology.orchestra.run.smithi189.stdout:Total 629 kB/s | 270 kB 00:00 2024-09-11T08:31:41.342 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:31:41.351 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:31:41.351 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:31:41.381 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:31:41.382 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:31:41.594 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:31:41.625 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install libcephfs2 2024-09-11T08:31:41.668 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:31:41.685 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:31:41.685 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:31:41.685 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T08:31:41.685 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:42.196 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:42.288 INFO:teuthology.orchestra.run.smithi131.stdout:Package libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:31:42.331 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:42.332 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-11T08:31:42.332 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:42.413 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install libcephfs-devel 2024-09-11T08:31:42.676 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T08:31:42.676 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:42.676 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:31:42.676 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T08:31:42.676 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:42.677 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:42.949 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install librados-devel 2024-09-11T08:31:42.981 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:43.114 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:43.115 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:43.115 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repo Size 2024-09-11T08:31:43.115 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:43.115 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:31:43.115 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T08:31:43.115 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:43.115 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:31:43.115 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:43.115 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-11T08:31:43.115 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:43.116 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 25 k 2024-09-11T08:31:43.116 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 110 k 2024-09-11T08:31:43.116 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:31:43.269 INFO:teuthology.orchestra.run.smithi131.stdout:libcephfs-devel-17.2.7-1623.gd0e6828a.el9.x86_6 163 kB/s | 25 kB 00:00 2024-09-11T08:31:43.270 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:43.270 INFO:teuthology.orchestra.run.smithi131.stdout:Total 162 kB/s | 25 kB 00:00 2024-09-11T08:31:43.270 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:31:43.273 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:31:43.273 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:31:43.275 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:31:43.275 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:31:43.361 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:31:43.474 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:43.527 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:39 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:43.666 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:43.667 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:43.667 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2024-09-11T08:31:43.667 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:43.667 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:31:43.667 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 126 k 2024-09-11T08:31:43.667 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:43.667 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:31:43.667 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:43.667 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-11T08:31:43.667 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:43.667 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 126 k 2024-09-11T08:31:43.667 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 448 k 2024-09-11T08:31:43.668 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:31:43.864 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:43.880 INFO:teuthology.orchestra.run.smithi189.stdout:librados-devel-17.2.7-1623.gd0e6828a.el9.x86_64 595 kB/s | 126 kB 00:00 2024-09-11T08:31:43.880 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:43.880 INFO:teuthology.orchestra.run.smithi189.stdout:Total 591 kB/s | 126 kB 00:00 2024-09-11T08:31:43.881 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:31:43.886 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:31:43.886 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:31:43.950 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:31:43.950 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:31:44.088 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:44.088 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:44.088 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:31:44.088 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:31:44.088 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:44.088 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:44.100 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:31:44.186 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:44.264 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install librados2 2024-09-11T08:31:44.673 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:44.825 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:44.914 INFO:teuthology.orchestra.run.smithi131.stdout:Package librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:31:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:44.935 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:31:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:31:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:44.936 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:44.957 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:44.958 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-11T08:31:44.958 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:45.023 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install librbd1 2024-09-11T08:31:45.234 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install libcephfs2 2024-09-11T08:31:45.581 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:39 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:45.671 INFO:teuthology.orchestra.run.smithi131.stdout:Package librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:31:45.714 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:45.714 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-11T08:31:45.715 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:45.789 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install python3-rados 2024-09-11T08:31:45.806 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:41 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:45.897 INFO:teuthology.orchestra.run.smithi189.stdout:Package libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:31:45.940 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:45.941 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-11T08:31:45.941 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:46.028 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install libcephfs-devel 2024-09-11T08:31:46.350 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:40 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:46.440 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:31:46.484 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:46.485 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-11T08:31:46.485 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:46.548 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install python3-rgw 2024-09-11T08:31:46.593 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:42 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:46.726 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:46.726 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:46.726 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2024-09-11T08:31:46.727 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:46.727 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:31:46.727 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T08:31:46.727 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:46.727 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:31:46.727 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:46.727 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-11T08:31:46.727 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:46.727 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 25 k 2024-09-11T08:31:46.727 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 110 k 2024-09-11T08:31:46.727 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:31:46.883 INFO:teuthology.orchestra.run.smithi189.stdout:libcephfs-devel-17.2.7-1623.gd0e6828a.el9.x86_6 161 kB/s | 25 kB 00:00 2024-09-11T08:31:46.883 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:46.883 INFO:teuthology.orchestra.run.smithi189.stdout:Total 159 kB/s | 25 kB 00:00 2024-09-11T08:31:46.883 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:31:46.887 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:31:46.887 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:31:46.888 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:31:46.889 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:31:46.978 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:31:47.087 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:47.102 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:41 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:47.192 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:31:47.234 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:47.235 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-11T08:31:47.235 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:47.294 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install python3-cephfs 2024-09-11T08:31:47.474 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:47.699 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:47.699 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:47.699 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:31:47.699 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:31:47.699 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:47.699 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:47.854 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:41 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:47.906 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install librados2 2024-09-11T08:31:47.944 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:31:47.987 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:47.988 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-11T08:31:47.988 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:48.060 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install python3-rbd 2024-09-11T08:31:48.473 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:44 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:48.564 INFO:teuthology.orchestra.run.smithi189.stdout:Package librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:31:48.606 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:48.607 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-11T08:31:48.607 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:48.622 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:42 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:48.686 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install librbd1 2024-09-11T08:31:48.713 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:31:48.755 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:48.756 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-11T08:31:48.756 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:48.823 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install rbd-fuse 2024-09-11T08:31:49.247 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:45 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:49.338 INFO:teuthology.orchestra.run.smithi189.stdout:Package librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:31:49.380 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:49.381 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-11T08:31:49.381 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:49.388 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:43 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:49.457 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install python3-rados 2024-09-11T08:31:49.522 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:49.522 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:49.522 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-09-11T08:31:49.523 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:49.523 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:31:49.523 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 82 k 2024-09-11T08:31:49.523 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:49.523 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:31:49.523 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:49.523 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-11T08:31:49.523 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:49.523 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 82 k 2024-09-11T08:31:49.523 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 222 k 2024-09-11T08:31:49.523 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:31:49.714 INFO:teuthology.orchestra.run.smithi131.stdout:rbd-fuse-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 431 kB/s | 82 kB 00:00 2024-09-11T08:31:49.714 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:49.714 INFO:teuthology.orchestra.run.smithi131.stdout:Total 428 kB/s | 82 kB 00:00 2024-09-11T08:31:49.714 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:31:49.721 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:31:49.721 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:31:49.785 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:31:49.785 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:31:49.943 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:31:50.022 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:46 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:50.044 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:50.112 INFO:teuthology.orchestra.run.smithi189.stdout:Package python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:31:50.155 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:50.156 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-11T08:31:50.157 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:50.233 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install python3-rgw 2024-09-11T08:31:50.412 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:50.679 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:50.679 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:50.679 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:31:50.679 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:31:50.679 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:50.679 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:50.790 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:46 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:50.880 INFO:teuthology.orchestra.run.smithi189.stdout:Package python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:31:50.913 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install rbd-mirror 2024-09-11T08:31:50.922 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:50.923 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-11T08:31:50.923 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:51.000 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install python3-cephfs 2024-09-11T08:31:51.492 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:45 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:51.572 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:47 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:51.634 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:51.635 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:51.635 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repo Size 2024-09-11T08:31:51.635 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:51.635 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:31:51.636 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T08:31:51.636 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:51.636 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:31:51.636 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:51.636 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-11T08:31:51.636 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:51.636 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 2.9 M 2024-09-11T08:31:51.636 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 12 M 2024-09-11T08:31:51.636 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:31:51.662 INFO:teuthology.orchestra.run.smithi189.stdout:Package python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:31:51.705 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:51.706 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-11T08:31:51.706 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:51.787 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install python3-rbd 2024-09-11T08:31:52.068 INFO:teuthology.orchestra.run.smithi131.stdout:rbd-mirror-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 6.8 MB/s | 2.9 MB 00:00 2024-09-11T08:31:52.069 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:52.069 INFO:teuthology.orchestra.run.smithi131.stdout:Total 6.8 MB/s | 2.9 MB 00:00 2024-09-11T08:31:52.070 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:31:52.084 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:31:52.084 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:31:52.178 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:31:52.178 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:31:52.360 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:48 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:52.458 INFO:teuthology.orchestra.run.smithi189.stdout:Package python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T08:31:52.504 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:52.505 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-11T08:31:52.505 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:52.609 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:31:52.627 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install rbd-fuse 2024-09-11T08:31:52.668 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:52.695 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:52.695 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:31:52.695 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T08:31:52.695 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:31:52.695 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:31:52.695 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:53.228 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:49 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:53.368 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:53.368 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:53.369 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-09-11T08:31:53.369 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:53.369 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:31:53.369 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 82 k 2024-09-11T08:31:53.369 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:53.369 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:31:53.369 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:53.369 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-11T08:31:53.369 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:53.369 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 82 k 2024-09-11T08:31:53.369 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 222 k 2024-09-11T08:31:53.369 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:31:53.552 INFO:teuthology.orchestra.run.smithi189.stdout:rbd-fuse-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 447 kB/s | 82 kB 00:00 2024-09-11T08:31:53.553 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:53.554 INFO:teuthology.orchestra.run.smithi189.stdout:Total 443 kB/s | 82 kB 00:00 2024-09-11T08:31:53.554 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:31:53.561 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:31:53.561 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:31:53.624 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:31:53.624 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:31:53.767 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:31:53.811 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:53.811 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:53.811 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:31:53.811 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:31:53.811 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:53.811 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:53.874 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:54.058 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install rbd-nbd 2024-09-11T08:31:54.259 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:54.517 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:54.517 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:54.517 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:31:54.517 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:31:54.517 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:54.517 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:54.636 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:48 ago on Wed 11 Sep 2024 08:30:06 AM UTC. 2024-09-11T08:31:54.769 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-11T08:31:54.770 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:54.770 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-09-11T08:31:54.770 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:54.770 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-11T08:31:54.770 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 165 k 2024-09-11T08:31:54.770 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:54.770 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-11T08:31:54.770 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-11T08:31:54.770 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-11T08:31:54.770 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:54.771 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 165 k 2024-09-11T08:31:54.771 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 477 k 2024-09-11T08:31:54.771 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-11T08:31:54.786 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install rbd-mirror 2024-09-11T08:31:54.972 INFO:teuthology.orchestra.run.smithi131.stdout:rbd-nbd-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 822 kB/s | 165 kB 00:00 2024-09-11T08:31:54.972 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:54.973 INFO:teuthology.orchestra.run.smithi131.stdout:Total 817 kB/s | 165 kB 00:00 2024-09-11T08:31:54.973 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-11T08:31:54.980 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-11T08:31:54.981 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-11T08:31:55.044 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-11T08:31:55.044 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-11T08:31:55.193 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-11T08:31:55.303 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:55.364 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:51 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:55.502 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:55.503 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:55.503 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2024-09-11T08:31:55.503 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:55.503 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:31:55.503 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T08:31:55.503 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:55.503 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:31:55.503 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:55.503 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-11T08:31:55.503 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:55.503 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 2.9 M 2024-09-11T08:31:55.504 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 12 M 2024-09-11T08:31:55.504 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:31:55.725 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:55.830 INFO:teuthology.orchestra.run.smithi189.stdout:rbd-mirror-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 9.0 MB/s | 2.9 MB 00:00 2024-09-11T08:31:55.831 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:55.831 INFO:teuthology.orchestra.run.smithi189.stdout:Total 9.0 MB/s | 2.9 MB 00:00 2024-09-11T08:31:55.831 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:31:55.840 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:31:55.840 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:31:55.921 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:31:55.921 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:31:56.000 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:56.000 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:56.000 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-11T08:31:56.001 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:31:56.001 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:31:56.001 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-11T08:31:56.244 DEBUG:teuthology.parallel:result is None 2024-09-11T08:31:56.348 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:31:56.506 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:56.532 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:56.533 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T08:31:56.533 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T08:31:56.533 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:31:56.533 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T08:31:56.533 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:57.507 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:57.507 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:57.507 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:31:57.507 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:31:57.507 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:57.508 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:57.733 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install rbd-nbd 2024-09-11T08:31:58.313 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:54 ago on Wed 11 Sep 2024 08:30:04 AM UTC. 2024-09-11T08:31:58.452 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-11T08:31:58.452 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:58.452 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-09-11T08:31:58.452 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:58.453 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-11T08:31:58.453 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 165 k 2024-09-11T08:31:58.453 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:58.453 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-11T08:31:58.453 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-11T08:31:58.453 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-11T08:31:58.453 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:58.453 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 165 k 2024-09-11T08:31:58.453 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 477 k 2024-09-11T08:31:58.453 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-11T08:31:58.685 INFO:teuthology.orchestra.run.smithi189.stdout:rbd-nbd-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 715 kB/s | 165 kB 00:00 2024-09-11T08:31:58.685 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:31:58.685 INFO:teuthology.orchestra.run.smithi189.stdout:Total 710 kB/s | 165 kB 00:00 2024-09-11T08:31:58.685 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-11T08:31:58.693 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-11T08:31:58.693 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-11T08:31:58.758 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-11T08:31:58.758 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-11T08:31:58.910 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-11T08:31:59.014 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:59.446 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:59.707 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T08:31:59.708 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:59.708 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-11T08:31:59.708 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T08:31:59.708 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:31:59.708 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-11T08:31:59.971 DEBUG:teuthology.parallel:result is None 2024-09-11T08:31:59.972 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:32:00.118 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T08:32:00.145 INFO:teuthology.orchestra.run.smithi131.stdout:17.2.7-1623.gd0e6828a.el9 2024-09-11T08:32:00.145 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1623.gd0e6828a.el9 2024-09-11T08:32:00.145 INFO:teuthology.task.install:The correct ceph version 17.2.7-1623.gd0e6828a is installed. 2024-09-11T08:32:00.147 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:32:00.288 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T08:32:00.311 INFO:teuthology.orchestra.run.smithi189.stdout:17.2.7-1623.gd0e6828a.el9 2024-09-11T08:32:00.311 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1623.gd0e6828a.el9 2024-09-11T08:32:00.311 INFO:teuthology.task.install:The correct ceph version 17.2.7-1623.gd0e6828a is installed. 2024-09-11T08:32:00.313 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T08:32:00.313 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-11T08:32:00.313 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T08:32:00.345 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-11T08:32:00.345 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T08:32:00.378 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T08:32:00.378 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-11T08:32:00.378 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T08:32:00.413 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T08:32:00.482 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-11T08:32:00.482 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T08:32:00.509 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T08:32:00.576 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T08:32:00.576 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-11T08:32:00.576 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T08:32:00.605 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T08:32:00.673 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-11T08:32:00.673 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T08:32:00.699 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T08:32:00.765 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T08:32:00.766 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-11T08:32:00.766 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T08:32:00.793 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T08:32:00.860 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-11T08:32:00.860 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T08:32:00.885 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T08:32:00.952 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T08:32:01.046 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'mon_bind_msgr2': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'cephadm_mode': 'cephadm-package'} 2024-09-11T08:32:01.047 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:32:01.047 INFO:tasks.cephadm:Cluster fsid is 50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:32:01.047 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T08:32:01.047 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.131:6789]', 'mon.c': '[v1:172.21.15.131:6790]', 'mon.b': '[v1:172.21.15.189:6789]'} 2024-09-11T08:32:01.048 INFO:tasks.cephadm:First mon is mon.a on smithi131 2024-09-11T08:32:01.048 INFO:tasks.cephadm:First mgr is y 2024-09-11T08:32:01.048 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T08:32:01.048 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname $(hostname -s) 2024-09-11T08:32:01.076 DEBUG:teuthology.orchestra.run.smithi189:> sudo hostname $(hostname -s) 2024-09-11T08:32:01.112 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 on all hosts... 2024-09-11T08:32:01.112 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T08:32:01.119 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T08:32:02.711 INFO:teuthology.orchestra.run.smithi131.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:32:02.747 INFO:teuthology.orchestra.run.smithi189.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:32:30.648 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-11T08:32:30.648 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:32:30.649 INFO:teuthology.orchestra.run.smithi131.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T08:32:30.649 INFO:teuthology.orchestra.run.smithi131.stdout: "repo_digests": [ 2024-09-11T08:32:30.649 INFO:teuthology.orchestra.run.smithi131.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T08:32:30.649 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-09-11T08:32:30.649 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-11T08:32:31.569 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-11T08:32:31.569 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:32:31.569 INFO:teuthology.orchestra.run.smithi189.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T08:32:31.569 INFO:teuthology.orchestra.run.smithi189.stdout: "repo_digests": [ 2024-09-11T08:32:31.569 INFO:teuthology.orchestra.run.smithi189.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T08:32:31.570 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-11T08:32:31.570 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-11T08:32:31.587 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph 2024-09-11T08:32:31.621 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph 2024-09-11T08:32:31.649 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /etc/ceph 2024-09-11T08:32:31.691 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /etc/ceph 2024-09-11T08:32:31.716 INFO:tasks.cephadm:Writing seed config... 2024-09-11T08:32:31.717 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T08:32:31.717 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T08:32:31.717 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-09-11T08:32:31.717 INFO:tasks.cephadm: override: [global] ms bind msgr2 = False 2024-09-11T08:32:31.717 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-11T08:32:31.717 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T08:32:31.718 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T08:32:31.718 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T08:32:31.718 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T08:32:31.718 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T08:32:31.718 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T08:32:31.718 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-11T08:32:31.718 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T08:32:31.749 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 50e4d0bc-7018-11ef-bcea-c7b262605968 mon election default strategy = 1 ms bind msgr2 = False ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-11T08:32:31.750 DEBUG:teuthology.orchestra.run.smithi131:mon.a> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T08:32:31.793 DEBUG:teuthology.orchestra.run.smithi131:mgr.y> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T08:32:31.836 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T08:32:31.836 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 -v bootstrap --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.131:6789]' --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:32:32.037 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-11T08:32:32.038 INFO:teuthology.orchestra.run.smithi131.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9', '-v', 'bootstrap', '--fsid', '50e4d0bc-7018-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-addrv', '[v1:172.21.15.131:6789]', '--skip-admin-label'] 2024-09-11T08:32:32.061 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 5.2.2 2024-09-11T08:32:32.062 INFO:teuthology.orchestra.run.smithi131.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T08:32:32.062 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying podman|docker is present... 2024-09-11T08:32:32.084 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 5.2.2 2024-09-11T08:32:32.084 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying lvm2 is present... 2024-09-11T08:32:32.084 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying time synchronization is in place... 2024-09-11T08:32:32.092 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T08:32:32.092 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T08:32:32.099 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T08:32:32.099 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-09-11T08:32:32.106 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout enabled 2024-09-11T08:32:32.112 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout active 2024-09-11T08:32:32.112 INFO:teuthology.orchestra.run.smithi131.stdout:Unit chronyd.service is enabled and running 2024-09-11T08:32:32.113 INFO:teuthology.orchestra.run.smithi131.stdout:Repeating the final host check... 2024-09-11T08:32:32.136 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 5.2.2 2024-09-11T08:32:32.136 INFO:teuthology.orchestra.run.smithi131.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T08:32:32.136 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl is present 2024-09-11T08:32:32.136 INFO:teuthology.orchestra.run.smithi131.stdout:lvcreate is present 2024-09-11T08:32:32.143 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T08:32:32.143 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T08:32:32.149 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T08:32:32.149 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-09-11T08:32:32.156 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout enabled 2024-09-11T08:32:32.162 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout active 2024-09-11T08:32:32.163 INFO:teuthology.orchestra.run.smithi131.stdout:Unit chronyd.service is enabled and running 2024-09-11T08:32:32.163 INFO:teuthology.orchestra.run.smithi131.stdout:Host looks OK 2024-09-11T08:32:32.163 INFO:teuthology.orchestra.run.smithi131.stdout:Cluster fsid: 50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:32:32.163 INFO:teuthology.orchestra.run.smithi131.stdout:Acquiring lock 139764526953232 on /run/cephadm/50e4d0bc-7018-11ef-bcea-c7b262605968.lock 2024-09-11T08:32:32.163 INFO:teuthology.orchestra.run.smithi131.stdout:Lock 139764526953232 acquired on /run/cephadm/50e4d0bc-7018-11ef-bcea-c7b262605968.lock 2024-09-11T08:32:32.163 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying IP 172.21.15.131 port 6789 ... 2024-09-11T08:32:32.163 INFO:teuthology.orchestra.run.smithi131.stdout:Base mon IP(s) is [172.21.15.131:6789], mon addrv is [v1:172.21.15.131:6789] 2024-09-11T08:32:32.167 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.131 metric 100 2024-09-11T08:32:32.167 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.131 metric 100 2024-09-11T08:32:32.170 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T08:32:32.170 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-11T08:32:32.170 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-11T08:32:32.172 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T08:32:32.172 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T08:32:32.173 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T08:32:32.173 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-11T08:32:32.173 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:72f9/64 scope link noprefixroute 2024-09-11T08:32:32.173 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T08:32:32.173 INFO:teuthology.orchestra.run.smithi131.stdout:Mon IP `172.21.15.131` is in CIDR network `172.21.0.0/20` 2024-09-11T08:32:32.173 INFO:teuthology.orchestra.run.smithi131.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20'] 2024-09-11T08:32:32.173 INFO:teuthology.orchestra.run.smithi131.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T08:32:32.174 INFO:teuthology.orchestra.run.smithi131.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:32:32.640 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T08:32:32.640 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T08:32:32.640 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T08:32:32.640 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying blob sha256:0d2f264d63cec2f6fb0f85eed7b2c8c5b4db3ab9edd9d71a74c04647294a553f 2024-09-11T08:32:32.640 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T08:32:32.640 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying config sha256:4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T08:32:32.641 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T08:32:33.545 INFO:teuthology.orchestra.run.smithi131.stdout:ceph: stdout ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T08:32:33.545 INFO:teuthology.orchestra.run.smithi131.stdout:Ceph version: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T08:32:33.545 INFO:teuthology.orchestra.run.smithi131.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T08:32:34.475 INFO:teuthology.orchestra.run.smithi131.stdout:stat: stdout 167 167 2024-09-11T08:32:34.475 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial keys... 2024-09-11T08:32:37.133 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQCiVeFmuUMUMBAAIZg6yOKL8c9IpfYt+kKf3Q== 2024-09-11T08:32:46.783 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQCoVeFmVd2CDRAATeNFQ7yEUx6Zh+LqVkMI4A== 2024-09-11T08:32:49.192 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQCwVeFmMBbvHBAAkzx1Q37DR6t8rbT5j8SkmA== 2024-09-11T08:32:49.193 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial monmap... 2024-09-11T08:32:50.141 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T08:32:50.141 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-11T08:32:50.141 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:32:50.141 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T08:32:50.141 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool for a [v1:172.21.15.131:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T08:32:50.141 INFO:teuthology.orchestra.run.smithi131.stdout:setting min_mon_release = octopus 2024-09-11T08:32:50.141 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: set fsid to 50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:32:50.141 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T08:32:50.142 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:32:50.142 INFO:teuthology.orchestra.run.smithi131.stdout:Creating mon... 2024-09-11T08:32:51.080 INFO:teuthology.orchestra.run.smithi131.stdout:create mon.a on 2024-09-11T08:32:51.279 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T08:32:51.415 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T08:32:51.561 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-50e4d0bc-7018-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-50e4d0bc-7018-11ef-bcea-c7b262605968.target. 2024-09-11T08:32:51.561 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-50e4d0bc-7018-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-50e4d0bc-7018-11ef-bcea-c7b262605968.target. 2024-09-11T08:32:51.728 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mon.a 2024-09-11T08:32:51.728 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to reset failed state of unit ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T08:32:51.888 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-50e4d0bc-7018-11ef-bcea-c7b262605968.target.wants/ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@.service. 2024-09-11T08:32:52.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:52 smithi131 podman[29052]: 2024-09-11 08:32:52.039565268 +0000 UTC m=+0.015823735 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:32:52.654 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:32:52.654 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-09-11T08:32:52.662 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:32:52.662 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-09-11T08:32:52.662 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-09-11T08:32:52.662 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T08:32:52.662 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mon to start... 2024-09-11T08:32:52.663 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mon... 2024-09-11T08:32:52.684 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:52 smithi131 podman[29052]: 2024-09-11 08:32:52.539548652 +0000 UTC m=+0.515807110 container init 6ef82003acdc394574cb0e3f447c740aa9889be3c50b95e1130724ae59dab084 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T08:32:52.685 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:52 smithi131 podman[29052]: 2024-09-11 08:32:52.542791381 +0000 UTC m=+0.519049845 container start 6ef82003acdc394574cb0e3f447c740aa9889be3c50b95e1130724ae59dab084 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T08:32:52.685 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:52 smithi131 ceph-mon[29071]: mkfs 50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:32:52.685 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:52 smithi131 ceph-mon[29071]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T08:32:52.685 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:52 smithi131 bash[29052]: 6ef82003acdc394574cb0e3f447c740aa9889be3c50b95e1130724ae59dab084 2024-09-11T08:32:52.685 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:52 smithi131 systemd[1]: Started Ceph mon.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:32:53.796 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:53 smithi131 ceph-mon[29071]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T08:32:53.796 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:53 smithi131 ceph-mon[29071]: monmap e1: 1 mons at {a=v1:172.21.15.131:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:32:53.796 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:53 smithi131 ceph-mon[29071]: fsmap 2024-09-11T08:32:53.797 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:53 smithi131 ceph-mon[29071]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T08:32:53.797 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:53 smithi131 ceph-mon[29071]: mgrmap e1: no daemons active 2024-09-11T08:32:53.797 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:53 smithi131 ceph-mon[29071]: from='client.? v1:172.21.15.131:0/2030043589' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T08:32:53.798 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T08:32:53.800 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout id: 50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:32:53.800 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T08:32:53.800 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-11T08:32:53.800 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout services: 2024-09-11T08:32:53.800 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.597863s) 2024-09-11T08:32:53.800 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T08:32:53.800 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T08:32:53.801 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-11T08:32:53.801 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout data: 2024-09-11T08:32:53.801 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T08:32:53.801 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T08:32:53.801 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T08:32:53.801 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T08:32:53.801 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-11T08:32:53.801 INFO:teuthology.orchestra.run.smithi131.stdout:mon is available 2024-09-11T08:32:53.801 INFO:teuthology.orchestra.run.smithi131.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T08:32:55.049 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:54 smithi131 ceph-mon[29071]: from='client.? v1:172.21.15.131:0/772574388' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T08:32:55.049 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:54 smithi131 ceph-mon[29071]: from='client.? v1:172.21.15.131:0/772574388' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T08:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-11T08:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T08:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout fsid = 50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T08:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_host = [v1:172.21.15.131:6789] 2024-09-11T08:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T08:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T08:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T08:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T08:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-11T08:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T08:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T08:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-11T08:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T08:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T08:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T08:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout:Generating new minimal ceph.conf... 2024-09-11T08:32:55.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:55 smithi131 ceph-mon[29071]: from='client.? v1:172.21.15.131:0/3603415823' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:32:56.375 INFO:teuthology.orchestra.run.smithi131.stdout:Restarting the monitor... 2024-09-11T08:32:56.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:56 smithi131 systemd[1]: Stopping Ceph mon.a for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:32:56.942 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a[29066]: 2024-09-11T08:32:56.667+0000 7fd4f2868640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:32:56.942 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a[29066]: 2024-09-11T08:32:56.667+0000 7fd4f2868640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T08:32:56.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:56 smithi131 podman[29323]: 2024-09-11 08:32:56.718929069 +0000 UTC m=+0.170778966 container died 6ef82003acdc394574cb0e3f447c740aa9889be3c50b95e1130724ae59dab084 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-11T08:32:57.243 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:57 smithi131 podman[29323]: 2024-09-11 08:32:57.034357506 +0000 UTC m=+0.486207395 container cleanup 6ef82003acdc394574cb0e3f447c740aa9889be3c50b95e1130724ae59dab084 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T08:32:57.244 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:57 smithi131 bash[29323]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a 2024-09-11T08:32:57.620 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:57 smithi131 podman[29336]: 2024-09-11 08:32:57.243668047 +0000 UTC m=+0.525885548 container remove 6ef82003acdc394574cb0e3f447c740aa9889be3c50b95e1130724ae59dab084 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T08:32:57.620 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:57 smithi131 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T08:32:57.620 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:57 smithi131 systemd[1]: Stopped Ceph mon.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:32:57.620 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:57 smithi131 systemd[1]: Starting Ceph mon.a for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:32:57.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:57 smithi131 podman[29427]: 2024-09-11 08:32:57.705603098 +0000 UTC m=+0.015680624 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:32:58.196 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:57 smithi131 podman[29427]: 2024-09-11 08:32:57.903026131 +0000 UTC m=+0.213103702 container create 4f1b488f6614c3694ce35cf32141520f4df7d38fdd7a811583c46cf618e61859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:32:58.197 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 podman[29427]: 2024-09-11 08:32:58.113466446 +0000 UTC m=+0.423543966 container init 4f1b488f6614c3694ce35cf32141520f4df7d38fdd7a811583c46cf618e61859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-11T08:32:58.197 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 podman[29427]: 2024-09-11 08:32:58.116488965 +0000 UTC m=+0.426566481 container start 4f1b488f6614c3694ce35cf32141520f4df7d38fdd7a811583c46cf618e61859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-11T08:32:58.197 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T08:32:58.197 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 2 2024-09-11T08:32:58.197 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: pidfile_write: ignore empty --pid-file 2024-09-11T08:32:58.197 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: load: jerasure load: lrc 2024-09-11T08:32:58.197 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: RocksDB version: 6.15.5 2024-09-11T08:32:58.197 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T08:32:58.197 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Compile date Sep 10 2024 2024-09-11T08:32:58.198 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: DB SUMMARY 2024-09-11T08:32:58.198 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: DB Session ID: FOW21SNON1MK1I59ZPFJ 2024-09-11T08:32:58.198 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: CURRENT file: CURRENT 2024-09-11T08:32:58.198 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T08:32:58.198 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-11T08:32:58.198 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T08:32:58.198 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 82002 ; 2024-09-11T08:32:58.198 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.error_if_exists: 0 2024-09-11T08:32:58.198 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.create_if_missing: 0 2024-09-11T08:32:58.199 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.paranoid_checks: 1 2024-09-11T08:32:58.200 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T08:32:58.200 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.env: 0x55a921bc5440 2024-09-11T08:32:58.200 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.fs: Posix File System 2024-09-11T08:32:58.201 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.info_log: 0x55a9243cae60 2024-09-11T08:32:58.201 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T08:32:58.201 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.statistics: (nil) 2024-09-11T08:32:58.201 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.use_fsync: 0 2024-09-11T08:32:58.201 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_log_file_size: 0 2024-09-11T08:32:58.201 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T08:32:58.201 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T08:32:58.201 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T08:32:58.201 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T08:32:58.201 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.allow_fallocate: 1 2024-09-11T08:32:58.202 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T08:32:58.202 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T08:32:58.202 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.use_direct_reads: 0 2024-09-11T08:32:58.202 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T08:32:58.202 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T08:32:58.202 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.db_log_dir: 2024-09-11T08:32:58.202 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-11T08:32:58.203 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T08:32:58.203 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T08:32:58.203 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T08:32:58.203 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T08:32:58.203 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T08:32:58.203 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T08:32:58.203 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T08:32:58.203 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T08:32:58.203 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.write_buffer_manager: 0x55a924488780 2024-09-11T08:32:58.203 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T08:32:58.203 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T08:32:58.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T08:32:58.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T08:32:58.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T08:32:58.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T08:32:58.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T08:32:58.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T08:32:58.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T08:32:58.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.unordered_write: 0 2024-09-11T08:32:58.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T08:32:58.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T08:32:58.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T08:32:58.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T08:32:58.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.row_cache: None 2024-09-11T08:32:58.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.wal_filter: None 2024-09-11T08:32:58.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T08:32:58.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T08:32:58.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.preserve_deletes: 0 2024-09-11T08:32:58.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.two_write_queues: 0 2024-09-11T08:32:58.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T08:32:58.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.atomic_flush: 0 2024-09-11T08:32:58.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T08:32:58.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T08:32:58.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T08:32:58.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.log_readahead_size: 0 2024-09-11T08:32:58.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T08:32:58.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T08:32:58.206 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T08:32:58.206 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T08:32:58.206 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T08:32:58.206 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T08:32:58.206 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_background_jobs: 2 2024-09-11T08:32:58.206 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_background_compactions: -1 2024-09-11T08:32:58.206 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_subcompactions: 1 2024-09-11T08:32:58.206 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T08:32:58.206 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T08:32:58.206 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T08:32:58.206 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T08:32:58.206 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T08:32:58.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T08:32:58.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T08:32:58.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T08:32:58.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_open_files: -1 2024-09-11T08:32:58.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T08:32:58.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T08:32:58.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T08:32:58.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T08:32:58.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_background_flushes: -1 2024-09-11T08:32:58.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Compression algorithms supported: 2024-09-11T08:32:58.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: kZSTD supported: 0 2024-09-11T08:32:58.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: kXpressCompression supported: 0 2024-09-11T08:32:58.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: kBZip2Compression supported: 0 2024-09-11T08:32:58.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T08:32:58.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: kLZ4Compression supported: 1 2024-09-11T08:32:58.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: kZlibCompression supported: 1 2024-09-11T08:32:58.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T08:32:58.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: kSnappyCompression supported: 1 2024-09-11T08:32:58.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T08:32:58.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-11T08:32:58.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T08:32:58.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T08:32:58.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.merge_operator: 2024-09-11T08:32:58.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compaction_filter: None 2024-09-11T08:32:58.208 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T08:32:58.209 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T08:32:58.209 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T08:32:58.209 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T08:32:58.209 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a924371a38) 2024-09-11T08:32:58.209 INFO:journalctl@ceph.mon.a.smithi131.stdout: cache_index_and_filter_blocks: 1 2024-09-11T08:32:58.209 INFO:journalctl@ceph.mon.a.smithi131.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T08:32:58.209 INFO:journalctl@ceph.mon.a.smithi131.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T08:32:58.209 INFO:journalctl@ceph.mon.a.smithi131.stdout: pin_top_level_index_and_filter: 1 2024-09-11T08:32:58.209 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_type: 0 2024-09-11T08:32:58.209 INFO:journalctl@ceph.mon.a.smithi131.stdout: data_block_index_type: 0 2024-09-11T08:32:58.209 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_shortening: 1 2024-09-11T08:32:58.209 INFO:journalctl@ceph.mon.a.smithi131.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T08:32:58.209 INFO:journalctl@ceph.mon.a.smithi131.stdout: hash_index_allow_collision: 1 2024-09-11T08:32:58.209 INFO:journalctl@ceph.mon.a.smithi131.stdout: checksum: 1 2024-09-11T08:32:58.209 INFO:journalctl@ceph.mon.a.smithi131.stdout: no_block_cache: 0 2024-09-11T08:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache: 0x55a9243a71f0 2024-09-11T08:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_name: BinnedLRUCache 2024-09-11T08:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_options: 2024-09-11T08:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout: capacity : 536870912 2024-09-11T08:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout: num_shard_bits : 4 2024-09-11T08:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout: strict_capacity_limit : 0 2024-09-11T08:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout: high_pri_pool_ratio: 0.000 2024-09-11T08:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_compressed: (nil) 2024-09-11T08:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout: persistent_cache: (nil) 2024-09-11T08:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_size: 4096 2024-09-11T08:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_size_deviation: 10 2024-09-11T08:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_restart_interval: 16 2024-09-11T08:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_block_restart_interval: 1 2024-09-11T08:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout: metadata_block_size: 4096 2024-09-11T08:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout: partition_filters: 0 2024-09-11T08:32:58.210 INFO:journalctl@ceph.mon.a.smithi131.stdout: use_delta_encoding: 1 2024-09-11T08:32:58.211 INFO:journalctl@ceph.mon.a.smithi131.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T08:32:58.211 INFO:journalctl@ceph.mon.a.smithi131.stdout: whole_key_filtering: 1 2024-09-11T08:32:58.211 INFO:journalctl@ceph.mon.a.smithi131.stdout: verify_compression: 0 2024-09-11T08:32:58.211 INFO:journalctl@ceph.mon.a.smithi131.stdout: read_amp_bytes_per_bit: 0 2024-09-11T08:32:58.211 INFO:journalctl@ceph.mon.a.smithi131.stdout: format_version: 4 2024-09-11T08:32:58.211 INFO:journalctl@ceph.mon.a.smithi131.stdout: enable_index_compression: 1 2024-09-11T08:32:58.211 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_align: 0 2024-09-11T08:32:58.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T08:32:58.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T08:32:58.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compression: NoCompression 2024-09-11T08:32:58.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T08:32:58.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T08:32:58.211 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T08:32:58.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.num_levels: 7 2024-09-11T08:32:58.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T08:32:58.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T08:32:58.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T08:32:58.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T08:32:58.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T08:32:58.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T08:32:58.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T08:32:58.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:32:58.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T08:32:58.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T08:32:58.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T08:32:58.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T08:32:58.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T08:32:58.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T08:32:58.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:32:58.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T08:32:58.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T08:32:58.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T08:32:58.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T08:32:58.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T08:32:58.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T08:32:58.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T08:32:58.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T08:32:58.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T08:32:58.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T08:32:58.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T08:32:58.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T08:32:58.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T08:32:58.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T08:32:58.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T08:32:58.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T08:32:58.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T08:32:58.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T08:32:58.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T08:32:58.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T08:32:58.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T08:32:58.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T08:32:58.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T08:32:58.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T08:32:58.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T08:32:58.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T08:32:58.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T08:32:58.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T08:32:58.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T08:32:58.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T08:32:58.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T08:32:58.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T08:32:58.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T08:32:58.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T08:32:58.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T08:32:58.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.inplace_update_support: 0 2024-09-11T08:32:58.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T08:32:58.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T08:32:58.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T08:32:58.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T08:32:58.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.bloom_locality: 0 2024-09-11T08:32:58.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.max_successive_merges: 0 2024-09-11T08:32:58.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T08:32:58.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T08:32:58.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T08:32:58.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T08:32:58.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.ttl: 2592000 2024-09-11T08:32:58.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T08:32:58.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.enable_blob_files: false 2024-09-11T08:32:58.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.min_blob_size: 0 2024-09-11T08:32:58.218 INFO:teuthology.orchestra.run.smithi131.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T08:32:58.218 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T08:32:58.218 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T08:32:58.219 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T08:32:58.219 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T08:32:58.219 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T08:32:58.219 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-11T08:32:58.219 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-11T08:32:58.219 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043578140758, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-11T08:32:58.219 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-11T08:32:58.219 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T08:32:58.219 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043578141469, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 78601, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 76863, "index_size": 200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9969, "raw_average_key_size": 47, "raw_value_size": 71185, "raw_average_value_size": 337, "num_data_blocks": 10, "num_entries": 211, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726043578, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "fa504201-5b86-48c5-ba88-4ffdcf3424a8", "db_session_id": "FOW21SNON1MK1I59ZPFJ"}} 2024-09-11T08:32:58.219 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-11T08:32:58.219 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043578141833, "job": 1, "event": "recovery_finished"} 2024-09-11T08:32:58.220 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T08:32:58.220 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55a9243da700 2024-09-11T08:32:58.220 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: DB pointer 0x55a9243aa000 2024-09-11T08:32:58.220 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T08:32:58.220 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T08:32:58.220 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** DB Stats ** 2024-09-11T08:32:58.220 INFO:journalctl@ceph.mon.a.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:32:58.220 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T08:32:58.220 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T08:32:58.220 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:32:58.220 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T08:32:58.221 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T08:32:58.221 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:32:58.221 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-09-11T08:32:58.221 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-11T08:32:58.221 INFO:journalctl@ceph.mon.a.smithi131.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:32:58.221 INFO:journalctl@ceph.mon.a.smithi131.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:32:58.221 INFO:journalctl@ceph.mon.a.smithi131.stdout: L0 2/0 78.51 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.8 0.00 0.00 1 0.000 0 0 2024-09-11T08:32:58.221 INFO:journalctl@ceph.mon.a.smithi131.stdout: Sum 2/0 78.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.8 0.00 0.00 1 0.000 0 0 2024-09-11T08:32:58.221 INFO:journalctl@ceph.mon.a.smithi131.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.8 0.00 0.00 1 0.000 0 0 2024-09-11T08:32:58.221 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-09-11T08:32:58.221 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-11T08:32:58.222 INFO:journalctl@ceph.mon.a.smithi131.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:32:58.222 INFO:journalctl@ceph.mon.a.smithi131.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:32:58.222 INFO:journalctl@ceph.mon.a.smithi131.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 159.8 0.00 0.00 1 0.000 0 0 2024-09-11T08:32:58.222 INFO:journalctl@ceph.mon.a.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:32:58.222 INFO:journalctl@ceph.mon.a.smithi131.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:32:58.222 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:32:58.222 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:32:58.222 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:32:58.222 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:32:58.222 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative compaction: 0.00 GB write, 24.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:32:58.222 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval compaction: 0.00 GB write, 24.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:32:58.222 INFO:journalctl@ceph.mon.a.smithi131.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:32:58.223 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-09-11T08:32:58.223 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:32:58.223 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-09-11T08:32:58.223 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-11T08:32:58.223 INFO:journalctl@ceph.mon.a.smithi131.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:32:58.223 INFO:journalctl@ceph.mon.a.smithi131.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:32:58.223 INFO:journalctl@ceph.mon.a.smithi131.stdout: L0 2/0 78.51 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.8 0.00 0.00 1 0.000 0 0 2024-09-11T08:32:58.223 INFO:journalctl@ceph.mon.a.smithi131.stdout: Sum 2/0 78.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.8 0.00 0.00 1 0.000 0 0 2024-09-11T08:32:58.223 INFO:journalctl@ceph.mon.a.smithi131.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T08:32:58.223 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-09-11T08:32:58.223 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-11T08:32:58.223 INFO:journalctl@ceph.mon.a.smithi131.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:32:58.223 INFO:journalctl@ceph.mon.a.smithi131.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:32:58.223 INFO:journalctl@ceph.mon.a.smithi131.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 159.8 0.00 0.00 1 0.000 0 0 2024-09-11T08:32:58.223 INFO:journalctl@ceph.mon.a.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:32:58.224 INFO:journalctl@ceph.mon.a.smithi131.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:32:58.224 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:32:58.224 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:32:58.224 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:32:58.224 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:32:58.224 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative compaction: 0.00 GB write, 24.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:32:58.224 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:32:58.224 INFO:journalctl@ceph.mon.a.smithi131.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:32:58.224 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-09-11T08:32:58.224 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:32:58.224 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: starting mon.a rank 0 at public addrs v1:172.21.15.131:6789/0 at bind addrs v1:172.21.15.131:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:32:58.224 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: mon.a@-1(???) e1 preinit fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:32:58.224 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: mon.a@-1(???).mds e1 new map 2024-09-11T08:32:58.224 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: mon.a@-1(???).mds e1 print_map 2024-09-11T08:32:58.225 INFO:journalctl@ceph.mon.a.smithi131.stdout: e1 2024-09-11T08:32:58.225 INFO:journalctl@ceph.mon.a.smithi131.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T08:32:58.225 INFO:journalctl@ceph.mon.a.smithi131.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T08:32:58.225 INFO:journalctl@ceph.mon.a.smithi131.stdout: legacy client fscid: -1 2024-09-11T08:32:58.225 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-09-11T08:32:58.225 INFO:journalctl@ceph.mon.a.smithi131.stdout: No filesystems configured 2024-09-11T08:32:58.225 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T08:32:58.225 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:32:58.225 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:32:58.225 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:32:58.225 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T08:32:58.225 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 bash[29427]: 4f1b488f6614c3694ce35cf32141520f4df7d38fdd7a811583c46cf618e61859 2024-09-11T08:32:58.225 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 systemd[1]: Started Ceph mon.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:32:58.524 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T08:32:58.524 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: monmap e1: 1 mons at {a=v1:172.21.15.131:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:32:58.524 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: fsmap 2024-09-11T08:32:58.524 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T08:32:58.524 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:58 smithi131 ceph-mon[29446]: mgrmap e1: no daemons active 2024-09-11T08:32:59.189 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T08:32:59.190 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:32:59.190 INFO:teuthology.orchestra.run.smithi131.stdout:Creating mgr... 2024-09-11T08:32:59.190 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T08:32:59.191 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T08:32:59.367 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mgr.y 2024-09-11T08:32:59.368 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to reset failed state of unit ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-11T08:32:59.520 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-50e4d0bc-7018-11ef-bcea-c7b262605968.target.wants/ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@.service. 2024-09-11T08:32:59.641 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:32:59 smithi131 systemd[1]: Starting Ceph mgr.y for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:32:59.965 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:32:59 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2563557086' entity='client.admin' 2024-09-11T08:32:59.965 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:32:59 smithi131 podman[29659]: 2024-09-11 08:32:59.652149041 +0000 UTC m=+0.016077332 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:32:59.966 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:32:59 smithi131 podman[29659]: 2024-09-11 08:32:59.771354786 +0000 UTC m=+0.135283113 container create 353609dc3bc565043b3c1b060eb47c6f4881af5673056a4e0f5c5bf72adb7f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:32:59.966 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:32:59 smithi131 podman[29659]: 2024-09-11 08:32:59.965923557 +0000 UTC m=+0.329851836 container init 353609dc3bc565043b3c1b060eb47c6f4881af5673056a4e0f5c5bf72adb7f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0) 2024-09-11T08:33:00.079 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:33:00.079 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-09-11T08:33:00.087 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:33:00.088 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-09-11T08:33:00.088 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-09-11T08:33:00.088 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T08:33:00.096 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:33:00.096 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-09-11T08:33:00.104 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:33:00.104 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-09-11T08:33:00.104 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-09-11T08:33:00.104 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T08:33:00.104 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr to start... 2024-09-11T08:33:00.105 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr... 2024-09-11T08:33:00.229 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:32:59 smithi131 podman[29659]: 2024-09-11 08:32:59.968975877 +0000 UTC m=+0.332904155 container start 353609dc3bc565043b3c1b060eb47c6f4881af5673056a4e0f5c5bf72adb7f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-11T08:33:00.229 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:00 smithi131 bash[29659]: 353609dc3bc565043b3c1b060eb47c6f4881af5673056a4e0f5c5bf72adb7f5c 2024-09-11T08:33:00.229 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:00 smithi131 systemd[1]: Started Ceph mgr.y for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:33:00.229 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:00.081+0000 7f7d2f050ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:33:00.229 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:00.140+0000 7f7d2f050ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:33:00.229 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:00.203+0000 7f7d2f050ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:33:00.551 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:00.329+0000 7f7d2f050ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:33:00.551 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:00.471+0000 7f7d2f050ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:33:00.802 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:00 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/663854520' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T08:33:00.803 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:00.741+0000 7f7d2f050ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:33:00.803 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:00.802+0000 7f7d2f050ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:33:01.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:00.966+0000 7f7d2f050ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:33:01.193 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:01.193 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:33:01.193 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "50e4d0bc-7018-11ef-bcea-c7b262605968", 2024-09-11T08:33:01.193 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T08:33:01.193 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T08:33:01.193 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T08:33:01.194 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T08:33:01.194 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:01.194 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T08:33:01.194 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T08:33:01.194 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-09-11T08:33:01.194 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:01.194 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T08:33:01.194 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T08:33:01.194 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:01.194 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-11T08:33:01.194 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T08:33:01.194 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:01.194 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T08:33:01.194 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T08:33:01.194 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:01.195 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T08:33:01.195 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:01.195 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T08:33:01.195 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T08:33:01.195 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T08:33:01.198 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T08:33:01.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T08:33:01.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T08:33:01.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:01.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T08:33:01.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T08:33:01.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T08:33:01.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T08:33:01.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T08:33:01.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T08:33:01.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T08:33:01.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T08:33:01.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T08:33:01.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:01.199 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T08:33:01.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:01.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T08:33:01.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T08:33:01.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:01.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T08:33:01.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T08:33:01.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T08:33:01.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T08:33:01.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T08:33:01.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T08:33:01.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T08:33:01.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:01.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:33:01.200 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:01.201 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T08:33:01.201 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:01.201 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T08:32:52.579341+0000", 2024-09-11T08:33:01.201 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:33:01.201 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:01.201 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T08:33:01.201 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:33:01.201 INFO:teuthology.orchestra.run.smithi131.stdout:mgr not available, waiting (1/15)... 2024-09-11T08:33:01.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:01 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:01.807+0000 7f7d2f050ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:33:01.891 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:01 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:01.871+0000 7f7d2f050ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:33:02.850 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:02 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:02.566+0000 7f7d2f050ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:33:02.850 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:02 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:02.675+0000 7f7d2f050ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:33:02.850 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:02 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:02.792+0000 7f7d2f050ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:33:03.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:02 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:02.850+0000 7f7d2f050ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:33:03.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:02 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:02.973+0000 7f7d2f050ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:33:03.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:03 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:03.039+0000 7f7d2f050ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:33:03.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:03 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:03.095+0000 7f7d2f050ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:33:03.503 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:03 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:03.235+0000 7f7d2f050ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:33:03.504 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:03 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:03.297+0000 7f7d2f050ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:33:03.504 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:03 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:03.363+0000 7f7d2f050ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:33:03.504 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:03 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:03.445+0000 7f7d2f050ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:33:03.504 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:03 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:03.503+0000 7f7d2f050ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:33:03.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:03 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/232095676' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T08:33:03.891 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:03 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:03.561+0000 7f7d2f050ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:33:04.249 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:04.252 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:33:04.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "50e4d0bc-7018-11ef-bcea-c7b262605968", 2024-09-11T08:33:04.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T08:33:04.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T08:33:04.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T08:33:04.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T08:33:04.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:04.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T08:33:04.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T08:33:04.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-09-11T08:33:04.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:04.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T08:33:04.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T08:33:04.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:04.253 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-11T08:33:04.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T08:33:04.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:04.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T08:33:04.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T08:33:04.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:04.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T08:33:04.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:04.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T08:33:04.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T08:33:04.254 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T08:33:04.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T08:33:04.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T08:33:04.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T08:33:04.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:04.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T08:33:04.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T08:33:04.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T08:33:04.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T08:33:04.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T08:33:04.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T08:33:04.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T08:33:04.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T08:33:04.258 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T08:33:04.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:04.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T08:33:04.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:04.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T08:33:04.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T08:33:04.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:04.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T08:33:04.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T08:33:04.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T08:33:04.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T08:33:04.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T08:33:04.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T08:33:04.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T08:33:04.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:04.259 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:33:04.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:04.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T08:33:04.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:04.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T08:32:52.579341+0000", 2024-09-11T08:33:04.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:33:04.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:04.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T08:33:04.260 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:33:04.260 INFO:teuthology.orchestra.run.smithi131.stdout:mgr not available, waiting (2/15)... 2024-09-11T08:33:04.640 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:04.315+0000 7f7d2f050ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:33:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-mon[29446]: Activating manager daemon y 2024-09-11T08:33:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-mon[29446]: mgrmap e2: y(active, starting, since 0.00281807s) 2024-09-11T08:33:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-mon[29446]: from='mgr.14100 v1:172.21.15.131:0/2440239629' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:33:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-mon[29446]: from='mgr.14100 v1:172.21.15.131:0/2440239629' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:33:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-mon[29446]: from='mgr.14100 v1:172.21.15.131:0/2440239629' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:33:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-mon[29446]: from='mgr.14100 v1:172.21.15.131:0/2440239629' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:33:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-mon[29446]: from='mgr.14100 v1:172.21.15.131:0/2440239629' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:33:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-mon[29446]: from='mgr.14100 v1:172.21.15.131:0/2440239629' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:33:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-mon[29446]: from='mgr.14100 v1:172.21.15.131:0/2440239629' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:33:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-mon[29446]: from='mgr.14100 v1:172.21.15.131:0/2440239629' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:33:05.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-mon[29446]: Manager daemon y is now available 2024-09-11T08:33:05.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-mon[29446]: from='mgr.14100 v1:172.21.15.131:0/2440239629' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:33:05.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-mon[29446]: from='mgr.14100 v1:172.21.15.131:0/2440239629' entity='mgr.y' 2024-09-11T08:33:05.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-mon[29446]: from='mgr.14100 v1:172.21.15.131:0/2440239629' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:33:05.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-mon[29446]: from='mgr.14100 v1:172.21.15.131:0/2440239629' entity='mgr.y' 2024-09-11T08:33:05.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:04 smithi131 ceph-mon[29446]: from='mgr.14100 v1:172.21.15.131:0/2440239629' entity='mgr.y' 2024-09-11T08:33:06.597 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:06 smithi131 ceph-mon[29446]: mgrmap e3: y(active, since 1.00671s) 2024-09-11T08:33:07.533 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:07.533 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:33:07.533 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "50e4d0bc-7018-11ef-bcea-c7b262605968", 2024-09-11T08:33:07.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T08:33:07.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T08:33:07.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T08:33:07.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T08:33:07.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:07.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T08:33:07.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T08:33:07.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-09-11T08:33:07.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:07.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T08:33:07.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T08:33:07.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:07.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-11T08:33:07.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T08:33:07.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:07.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T08:33:07.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T08:33:07.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:07.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T08:33:07.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:07.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T08:33:07.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T08:33:07.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T08:33:07.539 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T08:33:07.539 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T08:33:07.539 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T08:33:07.539 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:07.539 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T08:33:07.539 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T08:33:07.539 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T08:33:07.539 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T08:33:07.539 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T08:33:07.540 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T08:33:07.540 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T08:33:07.540 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T08:33:07.540 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T08:33:07.540 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:07.540 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T08:33:07.540 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:07.540 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T08:33:07.540 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T08:33:07.540 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:07.540 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T08:33:07.540 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T08:33:07.540 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T08:33:07.540 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T08:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T08:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T08:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T08:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-11T08:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T08:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T08:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T08:32:52.579341+0000", 2024-09-11T08:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T08:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-11T08:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T08:33:07.541 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:33:07.542 INFO:teuthology.orchestra.run.smithi131.stdout:mgr is available 2024-09-11T08:33:07.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:07 smithi131 ceph-mon[29446]: mgrmap e4: y(active, since 2s) 2024-09-11T08:33:07.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:07 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2904779267' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T08:33:08.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:08 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3682554834' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T08:33:08.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:08.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T08:33:08.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout fsid = 50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:33:08.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T08:33:08.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T08:33:08.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T08:33:08.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T08:33:08.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T08:33:08.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:08.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T08:33:08.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T08:33:08.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-11T08:33:08.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T08:33:08.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T08:33:08.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T08:33:08.824 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling cephadm module... 2024-09-11T08:33:09.792 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:09 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3237723543' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T08:33:09.793 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:09 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ignoring --setuser ceph since I am not root 2024-09-11T08:33:09.793 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:09 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ignoring --setgroup ceph since I am not root 2024-09-11T08:33:09.793 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:09 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:09.595+0000 7f093620bec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:33:09.793 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:09 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:09.649+0000 7f093620bec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:33:09.793 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:09 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:09.714+0000 7f093620bec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:33:10.102 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:09 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:09.836+0000 7f093620bec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:33:10.102 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:09 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:09.970+0000 7f093620bec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:33:10.354 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:10 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:10.232+0000 7f093620bec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:33:10.354 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:10 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:10.292+0000 7f093620bec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:33:10.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:10 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3237723543' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T08:33:10.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:10 smithi131 ceph-mon[29446]: mgrmap e5: y(active, since 5s) 2024-09-11T08:33:10.715 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:10 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:10.457+0000 7f093620bec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:33:11.304 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:33:11.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T08:33:11.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T08:33:11.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T08:33:11.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T08:33:11.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:33:11.305 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for the mgr to restart... 2024-09-11T08:33:11.305 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr epoch 5... 2024-09-11T08:33:11.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:11 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/252117956' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T08:33:11.641 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:11.308+0000 7f093620bec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:33:11.641 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:11.387+0000 7f093620bec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:33:12.387 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:12.102+0000 7f093620bec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:33:12.387 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:12.211+0000 7f093620bec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:33:12.387 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:12.329+0000 7f093620bec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:33:12.388 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:12.387+0000 7f093620bec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:33:12.640 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:12.510+0000 7f093620bec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:33:12.641 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:12.575+0000 7f093620bec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:33:12.641 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:12.632+0000 7f093620bec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:33:13.024 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:12.764+0000 7f093620bec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:33:13.025 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:12.826+0000 7f093620bec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:33:13.025 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:12.889+0000 7f093620bec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:33:13.025 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:12.968+0000 7f093620bec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:33:13.025 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:13.024+0000 7f093620bec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:33:13.391 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:13.080+0000 7f093620bec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:33:14.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:13.811+0000 7f093620bec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:33:14.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-mon[29446]: Active manager daemon y restarted 2024-09-11T08:33:14.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-mon[29446]: Activating manager daemon y 2024-09-11T08:33:14.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-mon[29446]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T08:33:14.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-mon[29446]: mgrmap e6: y(active, starting, since 0.00315717s) 2024-09-11T08:33:14.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:33:14.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:33:14.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:33:14.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:33:14.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:33:14.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-mon[29446]: Manager daemon y is now available 2024-09-11T08:33:14.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' 2024-09-11T08:33:14.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' 2024-09-11T08:33:14.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:33:14.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:33:14.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:33:14.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:13 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:33:15.131 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:14 smithi131 ceph-mon[29446]: Found migration_current of "None". Setting to last migration. 2024-09-11T08:33:15.132 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:14 smithi131 ceph-mon[29446]: mgrmap e7: y(active, since 1.00516s) 2024-09-11T08:33:15.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:33:15.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T08:33:15.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T08:33:15.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:33:15.377 INFO:teuthology.orchestra.run.smithi131.stdout:mgr epoch 5 is available 2024-09-11T08:33:15.377 INFO:teuthology.orchestra.run.smithi131.stdout:Setting orchestrator backend to cephadm... 2024-09-11T08:33:16.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:15 smithi131 ceph-mon[29446]: from='client.14122 v1:172.21.15.131:0/1537253430' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T08:33:16.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:15 smithi131 ceph-mon[29446]: from='client.14122 v1:172.21.15.131:0/1537253430' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T08:33:16.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-mon[29446]: mgrmap e8: y(active, since 2s) 2024-09-11T08:33:16.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-mon[29446]: from='client.14130 v1:172.21.15.131:0/2291106411' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:33:16.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' 2024-09-11T08:33:16.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:33:16.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' 2024-09-11T08:33:16.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:33:16.985 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:16.650+0000 7f08e85b9640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:33:16] ENGINE Error in HTTPServer.serve 2024-09-11T08:33:16.986 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: Traceback (most recent call last): 2024-09-11T08:33:16.986 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:33:16.986 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: self._connections.run(self.expiration_interval) 2024-09-11T08:33:16.986 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:33:16.986 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: self._run(expiration_interval) 2024-09-11T08:33:16.986 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:33:16.986 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:33:16.986 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:33:16.986 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:33:16.986 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:33:16.986 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: s = self.context.wrap_socket( 2024-09-11T08:33:16.986 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:33:16.986 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: return self.sslsocket_class._create( 2024-09-11T08:33:16.986 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:33:16.987 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: self.do_handshake() 2024-09-11T08:33:16.987 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:33:16.987 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: self._sslobj.do_handshake() 2024-09-11T08:33:16.987 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:33:16.987 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:16 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:17.608 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T08:33:17.608 INFO:teuthology.orchestra.run.smithi131.stdout:Generating ssh key... 2024-09-11T08:33:17.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:17 smithi131 ceph-mon[29446]: [11/Sep/2024:08:33:16] ENGINE Bus STARTING 2024-09-11T08:33:17.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:17 smithi131 ceph-mon[29446]: [11/Sep/2024:08:33:16] ENGINE Serving on https://172.21.15.131:7150 2024-09-11T08:33:17.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:17 smithi131 ceph-mon[29446]: [11/Sep/2024:08:33:16] ENGINE Bus STARTED 2024-09-11T08:33:17.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:17 smithi131 ceph-mon[29446]: [11/Sep/2024:08:33:16] ENGINE Error in HTTPServer.serve 2024-09-11T08:33:17.992 INFO:journalctl@ceph.mon.a.smithi131.stdout: Traceback (most recent call last): 2024-09-11T08:33:17.992 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:33:17.992 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:33:17.992 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:33:17.992 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._run(expiration_interval) 2024-09-11T08:33:17.992 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:33:17.992 INFO:journalctl@ceph.mon.a.smithi131.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:33:17.993 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:33:17.993 INFO:journalctl@ceph.mon.a.smithi131.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:33:17.993 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:33:17.993 INFO:journalctl@ceph.mon.a.smithi131.stdout: s = self.context.wrap_socket( 2024-09-11T08:33:17.993 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:33:17.993 INFO:journalctl@ceph.mon.a.smithi131.stdout: return self.sslsocket_class._create( 2024-09-11T08:33:17.993 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:33:17.993 INFO:journalctl@ceph.mon.a.smithi131.stdout: self.do_handshake() 2024-09-11T08:33:17.993 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:33:17.993 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._sslobj.do_handshake() 2024-09-11T08:33:17.993 INFO:journalctl@ceph.mon.a.smithi131.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:33:17.993 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:17 smithi131 ceph-mon[29446]: from='client.14132 v1:172.21.15.131:0/2225832999' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:33:18.596 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: Generating public/private rsa key pair. 2024-09-11T08:33:18.596 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: Your identification has been saved in /tmp/tmp274b28ae/key 2024-09-11T08:33:18.596 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: Your public key has been saved in /tmp/tmp274b28ae/key.pub 2024-09-11T08:33:18.596 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: The key fingerprint is: 2024-09-11T08:33:18.596 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: SHA256:CMCktYqMeYsNoR0mjB5zx7ALmH9ZBNQeKCbwZM9CbmY ceph-50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:33:18.596 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: The key's randomart image is: 2024-09-11T08:33:18.596 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: +---[RSA 3072]----+ 2024-09-11T08:33:18.596 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: |ooBoo+. | 2024-09-11T08:33:18.596 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: |=X+==.o | 2024-09-11T08:33:18.596 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: |BBE+++.. | 2024-09-11T08:33:18.597 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: |*%=oo+.. | 2024-09-11T08:33:18.597 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: |Bo+.o . S | 2024-09-11T08:33:18.597 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: | = o | 2024-09-11T08:33:18.597 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: |. o | 2024-09-11T08:33:18.597 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: | | 2024-09-11T08:33:18.597 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: | | 2024-09-11T08:33:18.597 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: +----[SHA256]-----+ 2024-09-11T08:33:19.416 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:19 smithi131 ceph-mon[29446]: from='client.14134 v1:172.21.15.131:0/3160882262' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:33:19.417 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:19 smithi131 ceph-mon[29446]: Generating ssh key... 2024-09-11T08:33:19.417 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:19 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' 2024-09-11T08:33:19.417 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:19 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' 2024-09-11T08:33:20.039 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCa8+jJPx+iyDSPV6JwozA6BhnvkGpZoOUMSx8xJJWGJ5in0X5pU1YtM4KG9hNgkO+Ln1zQkBCfQNbj4XCqPb4L4qZEGFjwtOvCuahbe/il56SIq41m1IJG8popGCSLJTvYFidKYNnJU2Lou00HkmDK3wkXkhgs8sE1Qi/ZILHet+n9BedHT4JZckodd7upTWTi34GA8VwtUv8MVGO0Lt0N+zdhJvnCT+KhXuDPAIFW1twj9lWyynwvVnXFVuJG3zvW3LXQcMP5NQk20Zw42JrP/4rkWXc9TLb213Cfp5Y3v/NUX0WlduqUDNF+TvZHhWVa8jKt8zKv67iCaR4WbT5xHvEbcrkD6AuhuzrDTCdKQjNLuYjUIlBAsrifstWtCpWPJN2xktwQVhPzLIT+A50yficNW7nuxeogAKXf8JEtxyKIIiFTJpgLeBGpAai9KmWgxo5rDs0Z/ZzD2nnMspsVZz7GG1gSxDQQ5VJLq3XrhDwusltUGr2ND2oU2liUAI0= ceph-50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:33:20.039 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T08:33:20.039 INFO:teuthology.orchestra.run.smithi131.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T08:33:20.049 INFO:teuthology.orchestra.run.smithi131.stdout:Adding host smithi131... 2024-09-11T08:33:20.381 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:20 smithi131 ceph-mon[29446]: from='client.14136 v1:172.21.15.131:0/613725452' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:33:21.675 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:21 smithi131 ceph-mon[29446]: from='client.14138 v1:172.21.15.131:0/2536070530' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi131", "addr": "172.21.15.131", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:33:22.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:22 smithi131 ceph-mon[29446]: Deploying cephadm binary to smithi131 2024-09-11T08:33:23.931 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Added host 'smithi131' with addr '172.21.15.131' 2024-09-11T08:33:23.932 INFO:teuthology.orchestra.run.smithi131.stdout:Deploying unmanaged mon service... 2024-09-11T08:33:24.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:24 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' 2024-09-11T08:33:24.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:24 smithi131 ceph-mon[29446]: Added host smithi131 2024-09-11T08:33:24.509 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:24 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:33:25.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T08:33:25.308 INFO:teuthology.orchestra.run.smithi131.stdout:Deploying unmanaged mgr service... 2024-09-11T08:33:25.566 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:25 smithi131 ceph-mon[29446]: from='client.14140 v1:172.21.15.131:0/2225623621' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:33:25.566 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:25 smithi131 ceph-mon[29446]: Saving service mon spec with placement count:5 2024-09-11T08:33:25.566 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:25 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' 2024-09-11T08:33:26.601 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T08:33:27.370 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:26 smithi131 ceph-mon[29446]: from='client.14142 v1:172.21.15.131:0/951550051' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:33:27.370 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:26 smithi131 ceph-mon[29446]: Saving service mgr spec with placement count:2 2024-09-11T08:33:27.370 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:26 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' 2024-09-11T08:33:28.477 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:28 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' 2024-09-11T08:33:28.477 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:28 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' 2024-09-11T08:33:28.477 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:28 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2410545231' entity='client.admin' 2024-09-11T08:33:30.288 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:30 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/395660582' entity='client.admin' 2024-09-11T08:33:30.288 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:30 smithi131 ceph-mon[29446]: from='mgr.14118 v1:172.21.15.131:0/3283636746' entity='mgr.y' 2024-09-11T08:33:30.290 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling the dashboard module... 2024-09-11T08:33:31.317 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:31 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3669544245' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T08:33:31.317 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:31 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ignoring --setuser ceph since I am not root 2024-09-11T08:33:31.318 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:31 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ignoring --setgroup ceph since I am not root 2024-09-11T08:33:31.318 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:31 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:31.204+0000 7ffbf5407ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:33:31.318 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:31 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:31.258+0000 7ffbf5407ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:33:31.587 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:31 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:31.327+0000 7ffbf5407ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:33:31.587 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:31 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:31.452+0000 7ffbf5407ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:33:31.587 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:31 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:31.587+0000 7ffbf5407ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:33:32.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:32 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3669544245' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T08:33:32.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:32 smithi131 ceph-mon[29446]: mgrmap e9: y(active, since 17s) 2024-09-11T08:33:32.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:31 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:31.845+0000 7ffbf5407ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:33:32.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:31 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:31.904+0000 7ffbf5407ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:33:32.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:32 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:32.072+0000 7ffbf5407ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:33:32.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:33:32.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T08:33:32.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T08:33:32.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T08:33:32.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T08:33:32.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:33:32.949 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for the mgr to restart... 2024-09-11T08:33:32.949 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr epoch 9... 2024-09-11T08:33:33.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:33 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2211016318' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T08:33:33.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:32 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:32.955+0000 7ffbf5407ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:33:33.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:33 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:33.025+0000 7ffbf5407ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:33:34.012 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:33 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:33.729+0000 7ffbf5407ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:33:34.012 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:33 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:33.837+0000 7ffbf5407ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:33:34.012 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:33 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:33.954+0000 7ffbf5407ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:33:34.386 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:34 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:34.012+0000 7ffbf5407ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:33:34.387 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:34 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:34.135+0000 7ffbf5407ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:33:34.387 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:34 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:34.199+0000 7ffbf5407ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:33:34.387 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:34 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:34.255+0000 7ffbf5407ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:33:34.387 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:34 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:34.386+0000 7ffbf5407ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:33:34.640 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:34 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:34.449+0000 7ffbf5407ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:33:34.641 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:34 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:34.512+0000 7ffbf5407ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:33:34.641 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:34 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:34.590+0000 7ffbf5407ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:33:35.140 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:34 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:34.646+0000 7ffbf5407ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:33:35.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:34 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:34.702+0000 7ffbf5407ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:33:35.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:35 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:35.430+0000 7ffbf5407ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:33:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:35 smithi131 ceph-mon[29446]: Active manager daemon y restarted 2024-09-11T08:33:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:35 smithi131 ceph-mon[29446]: Activating manager daemon y 2024-09-11T08:33:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:35 smithi131 ceph-mon[29446]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T08:33:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:35 smithi131 ceph-mon[29446]: mgrmap e10: y(active, starting, since 0.00317081s) 2024-09-11T08:33:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:35 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:33:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:35 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:33:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:35 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:33:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:35 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:33:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:35 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:33:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:35 smithi131 ceph-mon[29446]: Manager daemon y is now available 2024-09-11T08:33:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:35 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:33:35.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:35 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:33:35.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:35 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:33:36.889 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:36.523+0000 7ffba5f64640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:33:36] ENGINE Error in HTTPServer.serve 2024-09-11T08:33:36.889 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: Traceback (most recent call last): 2024-09-11T08:33:36.889 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:33:36.889 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: self._connections.run(self.expiration_interval) 2024-09-11T08:33:36.889 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:33:36.889 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: self._run(expiration_interval) 2024-09-11T08:33:36.889 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:33:36.889 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:33:36.889 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:33:36.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:33:36.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:33:36.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: s = self.context.wrap_socket( 2024-09-11T08:33:36.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:33:36.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: return self.sslsocket_class._create( 2024-09-11T08:33:36.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:33:36.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: self.do_handshake() 2024-09-11T08:33:36.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:33:36.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: self._sslobj.do_handshake() 2024-09-11T08:33:36.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:33:36.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:33:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:33:37.150 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-09-11T08:33:37.151 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T08:33:37.151 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T08:33:37.151 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-09-11T08:33:37.151 INFO:teuthology.orchestra.run.smithi131.stdout:mgr epoch 9 is available 2024-09-11T08:33:37.151 INFO:teuthology.orchestra.run.smithi131.stdout:Generating a dashboard self-signed certificate... 2024-09-11T08:33:37.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:37 smithi131 ceph-mon[29446]: [11/Sep/2024:08:33:36] ENGINE Bus STARTING 2024-09-11T08:33:37.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:37 smithi131 ceph-mon[29446]: from='client.14154 v1:172.21.15.131:0/3844899113' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T08:33:37.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:37 smithi131 ceph-mon[29446]: mgrmap e11: y(active, since 1.00518s) 2024-09-11T08:33:37.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:37 smithi131 ceph-mon[29446]: from='client.14154 v1:172.21.15.131:0/3844899113' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T08:33:37.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:37 smithi131 ceph-mon[29446]: [11/Sep/2024:08:33:36] ENGINE Serving on https://172.21.15.131:7150 2024-09-11T08:33:37.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:37 smithi131 ceph-mon[29446]: [11/Sep/2024:08:33:36] ENGINE Bus STARTED 2024-09-11T08:33:37.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:37 smithi131 ceph-mon[29446]: [11/Sep/2024:08:33:36] ENGINE Error in HTTPServer.serve 2024-09-11T08:33:37.891 INFO:journalctl@ceph.mon.a.smithi131.stdout: Traceback (most recent call last): 2024-09-11T08:33:37.891 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:33:37.891 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:33:37.892 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:33:37.892 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._run(expiration_interval) 2024-09-11T08:33:37.892 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:33:37.892 INFO:journalctl@ceph.mon.a.smithi131.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:33:37.892 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:33:37.892 INFO:journalctl@ceph.mon.a.smithi131.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:33:37.892 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:33:37.892 INFO:journalctl@ceph.mon.a.smithi131.stdout: s = self.context.wrap_socket( 2024-09-11T08:33:37.892 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:33:37.892 INFO:journalctl@ceph.mon.a.smithi131.stdout: return self.sslsocket_class._create( 2024-09-11T08:33:37.892 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:33:37.892 INFO:journalctl@ceph.mon.a.smithi131.stdout: self.do_handshake() 2024-09-11T08:33:37.892 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:33:37.892 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._sslobj.do_handshake() 2024-09-11T08:33:37.893 INFO:journalctl@ceph.mon.a.smithi131.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:33:37.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:37 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:37.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:37 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:38.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T08:33:38.516 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial admin user... 2024-09-11T08:33:39.194 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:38 smithi131 ceph-mon[29446]: from='client.14162 v1:172.21.15.131:0/1117296079' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:33:39.194 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:38 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:39.194 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:38 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:39.194 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:38 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:39.195 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:38 smithi131 ceph-mon[29446]: mgrmap e12: y(active, since 2s) 2024-09-11T08:33:40.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:39 smithi131 ceph-mon[29446]: from='client.14164 v1:172.21.15.131:0/506811611' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:33:40.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:39 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:40.531 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$pyX7ycABgsRj/naPuQSWIeQmDjsj.w5x24zxsUxW9ljUkww2alz1y", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726043619, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T08:33:40.531 INFO:teuthology.orchestra.run.smithi131.stdout:Fetching dashboard port number... 2024-09-11T08:33:41.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:41 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2236929176' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T08:33:41.788 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T08:33:41.799 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T08:33:41.799 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-09-11T08:33:41.808 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T08:33:41.808 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-09-11T08:33:41.808 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-09-11T08:33:41.808 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T08:33:41.810 INFO:teuthology.orchestra.run.smithi131.stdout:Ceph Dashboard is now available at: 2024-09-11T08:33:41.810 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:33:41.810 INFO:teuthology.orchestra.run.smithi131.stdout: URL: https://smithi131.front.sepia.ceph.com:8443/ 2024-09-11T08:33:41.810 INFO:teuthology.orchestra.run.smithi131.stdout: User: admin 2024-09-11T08:33:41.810 INFO:teuthology.orchestra.run.smithi131.stdout: Password: adnrivm8xu 2024-09-11T08:33:41.811 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:33:41.811 INFO:teuthology.orchestra.run.smithi131.stdout:Saving cluster configuration to /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config directory 2024-09-11T08:33:41.811 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling autotune for osd_memory_target 2024-09-11T08:33:43.612 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:43 smithi131 ceph-mon[29446]: mgrmap e13: y(active, since 6s) 2024-09-11T08:33:44.211 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T08:33:44.211 INFO:teuthology.orchestra.run.smithi131.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T08:33:44.211 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:33:44.211 INFO:teuthology.orchestra.run.smithi131.stdout: sudo /sbin/cephadm shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:33:44.211 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:33:44.211 INFO:teuthology.orchestra.run.smithi131.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T08:33:44.211 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:33:44.211 INFO:teuthology.orchestra.run.smithi131.stdout: sudo /sbin/cephadm shell 2024-09-11T08:33:44.211 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:33:44.211 INFO:teuthology.orchestra.run.smithi131.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T08:33:44.212 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:33:44.212 INFO:teuthology.orchestra.run.smithi131.stdout: ceph telemetry on 2024-09-11T08:33:44.212 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:33:44.212 INFO:teuthology.orchestra.run.smithi131.stdout:For more information see: 2024-09-11T08:33:44.212 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:33:44.212 INFO:teuthology.orchestra.run.smithi131.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-11T08:33:44.212 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:33:44.212 INFO:teuthology.orchestra.run.smithi131.stdout:Bootstrap complete. 2024-09-11T08:33:44.248 INFO:tasks.cephadm:Fetching config... 2024-09-11T08:33:44.249 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-11T08:33:44.249 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T08:33:44.265 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T08:33:44.266 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-11T08:33:44.266 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T08:33:44.322 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T08:33:44.322 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-11T08:33:44.322 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T08:33:44.388 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T08:33:44.388 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-11T08:33:44.388 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T08:33:44.443 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T08:33:44.443 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCa8+jJPx+iyDSPV6JwozA6BhnvkGpZoOUMSx8xJJWGJ5in0X5pU1YtM4KG9hNgkO+Ln1zQkBCfQNbj4XCqPb4L4qZEGFjwtOvCuahbe/il56SIq41m1IJG8popGCSLJTvYFidKYNnJU2Lou00HkmDK3wkXkhgs8sE1Qi/ZILHet+n9BedHT4JZckodd7upTWTi34GA8VwtUv8MVGO0Lt0N+zdhJvnCT+KhXuDPAIFW1twj9lWyynwvVnXFVuJG3zvW3LXQcMP5NQk20Zw42JrP/4rkWXc9TLb213Cfp5Y3v/NUX0WlduqUDNF+TvZHhWVa8jKt8zKv67iCaR4WbT5xHvEbcrkD6AuhuzrDTCdKQjNLuYjUIlBAsrifstWtCpWPJN2xktwQVhPzLIT+A50yficNW7nuxeogAKXf8JEtxyKIIiFTJpgLeBGpAai9KmWgxo5rDs0Z/ZzD2nnMspsVZz7GG1gSxDQQ5VJLq3XrhDwusltUGr2ND2oU2liUAI0= ceph-50e4d0bc-7018-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T08:33:44.516 INFO:teuthology.orchestra.run.smithi131.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCa8+jJPx+iyDSPV6JwozA6BhnvkGpZoOUMSx8xJJWGJ5in0X5pU1YtM4KG9hNgkO+Ln1zQkBCfQNbj4XCqPb4L4qZEGFjwtOvCuahbe/il56SIq41m1IJG8popGCSLJTvYFidKYNnJU2Lou00HkmDK3wkXkhgs8sE1Qi/ZILHet+n9BedHT4JZckodd7upTWTi34GA8VwtUv8MVGO0Lt0N+zdhJvnCT+KhXuDPAIFW1twj9lWyynwvVnXFVuJG3zvW3LXQcMP5NQk20Zw42JrP/4rkWXc9TLb213Cfp5Y3v/NUX0WlduqUDNF+TvZHhWVa8jKt8zKv67iCaR4WbT5xHvEbcrkD6AuhuzrDTCdKQjNLuYjUIlBAsrifstWtCpWPJN2xktwQVhPzLIT+A50yficNW7nuxeogAKXf8JEtxyKIIiFTJpgLeBGpAai9KmWgxo5rDs0Z/ZzD2nnMspsVZz7GG1gSxDQQ5VJLq3XrhDwusltUGr2ND2oU2liUAI0= ceph-50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:33:44.527 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCa8+jJPx+iyDSPV6JwozA6BhnvkGpZoOUMSx8xJJWGJ5in0X5pU1YtM4KG9hNgkO+Ln1zQkBCfQNbj4XCqPb4L4qZEGFjwtOvCuahbe/il56SIq41m1IJG8popGCSLJTvYFidKYNnJU2Lou00HkmDK3wkXkhgs8sE1Qi/ZILHet+n9BedHT4JZckodd7upTWTi34GA8VwtUv8MVGO0Lt0N+zdhJvnCT+KhXuDPAIFW1twj9lWyynwvVnXFVuJG3zvW3LXQcMP5NQk20Zw42JrP/4rkWXc9TLb213Cfp5Y3v/NUX0WlduqUDNF+TvZHhWVa8jKt8zKv67iCaR4WbT5xHvEbcrkD6AuhuzrDTCdKQjNLuYjUIlBAsrifstWtCpWPJN2xktwQVhPzLIT+A50yficNW7nuxeogAKXf8JEtxyKIIiFTJpgLeBGpAai9KmWgxo5rDs0Z/ZzD2nnMspsVZz7GG1gSxDQQ5VJLq3XrhDwusltUGr2ND2oU2liUAI0= ceph-50e4d0bc-7018-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T08:33:44.638 INFO:teuthology.orchestra.run.smithi189.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCa8+jJPx+iyDSPV6JwozA6BhnvkGpZoOUMSx8xJJWGJ5in0X5pU1YtM4KG9hNgkO+Ln1zQkBCfQNbj4XCqPb4L4qZEGFjwtOvCuahbe/il56SIq41m1IJG8popGCSLJTvYFidKYNnJU2Lou00HkmDK3wkXkhgs8sE1Qi/ZILHet+n9BedHT4JZckodd7upTWTi34GA8VwtUv8MVGO0Lt0N+zdhJvnCT+KhXuDPAIFW1twj9lWyynwvVnXFVuJG3zvW3LXQcMP5NQk20Zw42JrP/4rkWXc9TLb213Cfp5Y3v/NUX0WlduqUDNF+TvZHhWVa8jKt8zKv67iCaR4WbT5xHvEbcrkD6AuhuzrDTCdKQjNLuYjUIlBAsrifstWtCpWPJN2xktwQVhPzLIT+A50yficNW7nuxeogAKXf8JEtxyKIIiFTJpgLeBGpAai9KmWgxo5rDs0Z/ZzD2nnMspsVZz7GG1gSxDQQ5VJLq3XrhDwusltUGr2ND2oU2liUAI0= ceph-50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:33:44.651 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T08:33:44.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:44 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/157644262' entity='client.admin' 2024-09-11T08:33:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:33:47.681 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:47 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3231894270' entity='client.admin' 2024-09-11T08:33:47.681 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:47 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:47.681 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:47 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:47.681 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:47 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:33:47.681 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:47 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:47.681 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:47 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:33:47.681 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:47 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:47.705 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T08:33:47.706 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T08:33:47.942 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:33:50.424 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi189 2024-09-11T08:33:50.424 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-11T08:33:50.425 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.conf 2024-09-11T08:33:50.444 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-11T08:33:50.444 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:33:50.502 INFO:tasks.cephadm:Adding host smithi189 to orchestrator... 2024-09-11T08:33:50.503 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch host add smithi189 2024-09-11T08:33:50.738 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:33:50.949 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:50 smithi131 ceph-mon[29446]: from='client.14174 v1:172.21.15.131:0/3161699323' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:33:50.950 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:50 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:50.950 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:50 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:33:50.950 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:50 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:33:50.950 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:50 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:33:50.950 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:50 smithi131 ceph-mon[29446]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-11T08:33:50.950 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:50 smithi131 ceph-mon[29446]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:33:50.950 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:50 smithi131 ceph-mon[29446]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:33:50.951 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:50 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:50.951 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:50 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:50.951 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:50 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:51.822 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:51 smithi131 ceph-mon[29446]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:33:53.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:53 smithi131 ceph-mon[29446]: from='client.14176 v1:172.21.15.131:0/3044883887' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:33:54.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:54 smithi131 ceph-mon[29446]: Deploying cephadm binary to smithi189 2024-09-11T08:33:55.191 INFO:teuthology.orchestra.run.smithi131.stdout:Added host 'smithi189' with addr '172.21.15.189' 2024-09-11T08:33:55.804 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T08:33:56.046 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:33:56.432 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:56.432 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:56 smithi131 ceph-mon[29446]: Added host smithi189 2024-09-11T08:33:56.432 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:33:56.433 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:56 smithi131 ceph-mon[29446]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:33:56.433 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:56.433 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:57.803 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:33:57.803 INFO:teuthology.orchestra.run.smithi131.stdout:[{"addr": "172.21.15.131", "hostname": "smithi131", "labels": [], "status": ""}, {"addr": "172.21.15.189", "hostname": "smithi189", "labels": [], "status": ""}] 2024-09-11T08:33:57.803 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:57 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:33:58.394 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T08:33:58.394 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T08:33:58.631 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:33:58.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:58 smithi131 ceph-mon[29446]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:33:58.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:33:58 smithi131 ceph-mon[29446]: from='client.14178 v1:172.21.15.131:0/3300733325' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:34:00.290 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:00 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:00.290 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:00 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:00.290 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:00 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:00.290 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:00 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:00.290 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:00 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:34:00.290 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:00 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:00.290 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:00 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:00.290 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:00 smithi131 ceph-mon[29446]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-11T08:34:00.291 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:00 smithi131 ceph-mon[29446]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:00.291 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:00 smithi131 ceph-mon[29446]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:34:00.291 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:00 smithi131 ceph-mon[29446]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:34:00.291 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:00 smithi131 ceph-mon[29446]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:34:00.291 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:00 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:00.291 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:00 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:00.291 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:00 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:01.251 INFO:teuthology.orchestra.run.smithi131.stderr:adjusted tunables profile to default 2024-09-11T08:34:01.541 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:01 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2597564181' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T08:34:01.807 INFO:tasks.cephadm:Adding mon.a on smithi131 2024-09-11T08:34:01.807 INFO:tasks.cephadm:Adding mon.c on smithi131 2024-09-11T08:34:01.807 INFO:tasks.cephadm:Adding mon.b on smithi189 2024-09-11T08:34:01.807 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi131:[v1:172.21.15.131:6789]=a;smithi131:[v1:172.21.15.131:6790]=c;smithi189:[v1:172.21.15.189:6789]=b' 2024-09-11T08:34:02.051 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:34:02.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:02 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2597564181' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T08:34:02.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:02 smithi131 ceph-mon[29446]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:34:02.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:02 smithi131 ceph-mon[29446]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:03.764 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mon update... 2024-09-11T08:34:04.311 DEBUG:teuthology.orchestra.run.smithi131:mon.c> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T08:34:04.314 DEBUG:teuthology.orchestra.run.smithi189:mon.b> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T08:34:04.317 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T08:34:04.317 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T08:34:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:04 smithi131 ceph-mon[29446]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:04 smithi131 ceph-mon[29446]: from='client.14182 v1:172.21.15.189:0/1894932059' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:[v1:172.21.15.131:6789]=a;smithi131:[v1:172.21.15.131:6790]=c;smithi189:[v1:172.21.15.189:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:04 smithi131 ceph-mon[29446]: Saving service mon spec with placement smithi131:[v1:172.21.15.131:6789]=a;smithi131:[v1:172.21.15.131:6790]=c;smithi189:[v1:172.21.15.189:6789]=b;count:3 2024-09-11T08:34:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:04 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:04 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:04 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:04 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:04 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:04 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:04 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:04 smithi131 ceph-mon[29446]: Deploying daemon mon.b on smithi189 2024-09-11T08:34:05.380 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:34:07.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:06 smithi131 ceph-mon[29446]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:08.076 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:34:08.076 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":1,"fsid":"50e4d0bc-7018-11ef-bcea-c7b262605968","modified":"2024-09-11T08:32:49.512160Z","created":"2024-09-11T08:32:49.512160Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T08:34:08.076 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 1 2024-09-11T08:34:08.232 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:08 smithi189 ceph-mon[32675]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-11T08:34:08.232 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:08 smithi189 bash[32644]: f858a635fc72da4510d87b28c757ec6bb7887c4ae42f4d72dd26197ec7f45a1b 2024-09-11T08:34:08.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:08 smithi189 systemd[1]: Started Ceph mon.b for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:34:09.836 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T08:34:09.837 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T08:34:10.098 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:34:13.444 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:34:13.444 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"50e4d0bc-7018-11ef-bcea-c7b262605968","modified":"2024-09-11T08:34:08.193542Z","created":"2024-09-11T08:32:49.512160Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T08:34:13.444 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-09-11T08:34:13.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:34:13.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:13.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: mon.a calling monitor election 2024-09-11T08:34:13.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:13.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:13.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:13.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: mon.b calling monitor election 2024-09-11T08:34:13.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:13.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:13.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:13.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:13.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T08:34:13.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: monmap e2: 2 mons at {a=v1:172.21.15.131:6789/0,b=v1:172.21.15.189:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:34:13.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: fsmap 2024-09-11T08:34:13.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:34:13.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: mgrmap e13: y(active, since 37s) 2024-09-11T08:34:13.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: overall HEALTH_OK 2024-09-11T08:34:13.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:13.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:13.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:13.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:13.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:13 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:13.672 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:34:13.672 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:13.672 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: mon.a calling monitor election 2024-09-11T08:34:13.672 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:13.672 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:13.673 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:13.673 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: mon.b calling monitor election 2024-09-11T08:34:13.673 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:13.673 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:13.673 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:13.673 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:13.673 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T08:34:13.673 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: monmap e2: 2 mons at {a=v1:172.21.15.131:6789/0,b=v1:172.21.15.189:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:34:13.673 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: fsmap 2024-09-11T08:34:13.673 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:34:13.673 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: mgrmap e13: y(active, since 37s) 2024-09-11T08:34:13.673 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: overall HEALTH_OK 2024-09-11T08:34:13.673 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:13.674 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:13.674 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:13.674 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:13.674 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:13 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:14.636 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:14 smithi131 ceph-mon[29446]: Deploying daemon mon.c on smithi131 2024-09-11T08:34:14.636 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:14 smithi131 ceph-mon[29446]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:14.636 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:14 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.189:0/4007520027' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:34:14.636 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:14 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:14.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:14 smithi189 ceph-mon[32675]: Deploying daemon mon.c on smithi131 2024-09-11T08:34:14.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:14 smithi189 ceph-mon[32675]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:14.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:14 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.189:0/4007520027' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:34:14.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:14 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:14.973 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T08:34:14.973 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T08:34:15.225 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:34:15.532 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:34:15 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:34:15.193+0000 7ffbd9999640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-11T08:34:16.168 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 podman[37179]: 2024-09-11 08:34:16.02862968 +0000 UTC m=+0.021778308 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:34:16.168 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 podman[37179]: 2024-09-11 08:34:16.168388935 +0000 UTC m=+0.161537563 container create 569d21b6765f851a50830410f022e15ec8da7ff7ab30888d1f4577303fc07168 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-c, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T08:34:16.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 podman[37179]: 2024-09-11 08:34:16.428648771 +0000 UTC m=+0.421797412 container init 569d21b6765f851a50830410f022e15ec8da7ff7ab30888d1f4577303fc07168 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-c, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, ceph=True) 2024-09-11T08:34:16.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 podman[37179]: 2024-09-11 08:34:16.432031391 +0000 UTC m=+0.425180020 container start 569d21b6765f851a50830410f022e15ec8da7ff7ab30888d1f4577303fc07168 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:34:16.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T08:34:16.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 2 2024-09-11T08:34:16.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: pidfile_write: ignore empty --pid-file 2024-09-11T08:34:16.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: load: jerasure load: lrc 2024-09-11T08:34:16.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: RocksDB version: 6.15.5 2024-09-11T08:34:16.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T08:34:16.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Compile date Sep 10 2024 2024-09-11T08:34:16.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: DB SUMMARY 2024-09-11T08:34:16.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: DB Session ID: SPSCLO8XZBUAIQGLLKC8 2024-09-11T08:34:16.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: CURRENT file: CURRENT 2024-09-11T08:34:16.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T08:34:16.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-11T08:34:16.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-11T08:34:16.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 566 ; 2024-09-11T08:34:16.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.error_if_exists: 0 2024-09-11T08:34:16.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.create_if_missing: 0 2024-09-11T08:34:16.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.paranoid_checks: 1 2024-09-11T08:34:16.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T08:34:16.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.env: 0x5652ca1e0440 2024-09-11T08:34:16.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.fs: Posix File System 2024-09-11T08:34:16.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.info_log: 0x5652cb3d2e80 2024-09-11T08:34:16.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T08:34:16.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.statistics: (nil) 2024-09-11T08:34:16.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.use_fsync: 0 2024-09-11T08:34:16.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_log_file_size: 0 2024-09-11T08:34:16.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T08:34:16.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T08:34:16.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T08:34:16.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T08:34:16.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.allow_fallocate: 1 2024-09-11T08:34:16.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T08:34:16.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T08:34:16.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.use_direct_reads: 0 2024-09-11T08:34:16.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T08:34:16.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T08:34:16.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.db_log_dir: 2024-09-11T08:34:16.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-09-11T08:34:16.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T08:34:16.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T08:34:16.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T08:34:16.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T08:34:16.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T08:34:16.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.write_buffer_manager: 0x5652cb490840 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.unordered_write: 0 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T08:34:16.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T08:34:16.896 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.row_cache: None 2024-09-11T08:34:16.896 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.wal_filter: None 2024-09-11T08:34:16.897 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T08:34:16.897 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T08:34:16.897 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.preserve_deletes: 0 2024-09-11T08:34:16.897 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.two_write_queues: 0 2024-09-11T08:34:16.897 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T08:34:16.897 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.atomic_flush: 0 2024-09-11T08:34:16.897 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T08:34:16.898 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T08:34:16.898 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T08:34:16.898 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.log_readahead_size: 0 2024-09-11T08:34:16.898 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T08:34:16.898 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T08:34:16.898 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T08:34:16.898 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T08:34:16.898 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T08:34:16.898 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T08:34:16.898 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_background_jobs: 2 2024-09-11T08:34:16.898 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_background_compactions: -1 2024-09-11T08:34:16.898 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_subcompactions: 1 2024-09-11T08:34:16.898 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T08:34:16.898 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T08:34:16.898 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T08:34:16.899 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T08:34:16.899 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T08:34:16.899 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T08:34:16.899 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T08:34:16.899 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T08:34:16.899 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_open_files: -1 2024-09-11T08:34:16.899 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T08:34:16.899 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T08:34:16.899 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T08:34:16.899 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T08:34:16.899 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_background_flushes: -1 2024-09-11T08:34:16.899 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Compression algorithms supported: 2024-09-11T08:34:16.899 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: kZSTD supported: 0 2024-09-11T08:34:16.899 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: kXpressCompression supported: 0 2024-09-11T08:34:16.899 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: kBZip2Compression supported: 0 2024-09-11T08:34:16.900 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T08:34:16.900 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: kLZ4Compression supported: 1 2024-09-11T08:34:16.900 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: kZlibCompression supported: 1 2024-09-11T08:34:16.900 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T08:34:16.900 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: kSnappyCompression supported: 1 2024-09-11T08:34:16.900 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T08:34:16.900 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2024-09-11T08:34:16.900 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T08:34:16.900 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T08:34:16.900 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.merge_operator: 2024-09-11T08:34:16.900 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compaction_filter: None 2024-09-11T08:34:16.900 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T08:34:16.900 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T08:34:16.901 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T08:34:16.901 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T08:34:16.901 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5652cb379a38) 2024-09-11T08:34:16.901 INFO:journalctl@ceph.mon.c.smithi131.stdout: cache_index_and_filter_blocks: 1 2024-09-11T08:34:16.901 INFO:journalctl@ceph.mon.c.smithi131.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T08:34:16.901 INFO:journalctl@ceph.mon.c.smithi131.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T08:34:16.901 INFO:journalctl@ceph.mon.c.smithi131.stdout: pin_top_level_index_and_filter: 1 2024-09-11T08:34:16.901 INFO:journalctl@ceph.mon.c.smithi131.stdout: index_type: 0 2024-09-11T08:34:16.901 INFO:journalctl@ceph.mon.c.smithi131.stdout: data_block_index_type: 0 2024-09-11T08:34:16.901 INFO:journalctl@ceph.mon.c.smithi131.stdout: index_shortening: 1 2024-09-11T08:34:16.901 INFO:journalctl@ceph.mon.c.smithi131.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T08:34:16.901 INFO:journalctl@ceph.mon.c.smithi131.stdout: hash_index_allow_collision: 1 2024-09-11T08:34:16.901 INFO:journalctl@ceph.mon.c.smithi131.stdout: checksum: 1 2024-09-11T08:34:16.901 INFO:journalctl@ceph.mon.c.smithi131.stdout: no_block_cache: 0 2024-09-11T08:34:16.901 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache: 0x5652cb3af1f0 2024-09-11T08:34:16.902 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache_name: BinnedLRUCache 2024-09-11T08:34:16.902 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache_options: 2024-09-11T08:34:16.902 INFO:journalctl@ceph.mon.c.smithi131.stdout: capacity : 536870912 2024-09-11T08:34:16.902 INFO:journalctl@ceph.mon.c.smithi131.stdout: num_shard_bits : 4 2024-09-11T08:34:16.902 INFO:journalctl@ceph.mon.c.smithi131.stdout: strict_capacity_limit : 0 2024-09-11T08:34:16.902 INFO:journalctl@ceph.mon.c.smithi131.stdout: high_pri_pool_ratio: 0.000 2024-09-11T08:34:16.902 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache_compressed: (nil) 2024-09-11T08:34:16.902 INFO:journalctl@ceph.mon.c.smithi131.stdout: persistent_cache: (nil) 2024-09-11T08:34:16.902 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_size: 4096 2024-09-11T08:34:16.902 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_size_deviation: 10 2024-09-11T08:34:16.902 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_restart_interval: 16 2024-09-11T08:34:16.902 INFO:journalctl@ceph.mon.c.smithi131.stdout: index_block_restart_interval: 1 2024-09-11T08:34:16.902 INFO:journalctl@ceph.mon.c.smithi131.stdout: metadata_block_size: 4096 2024-09-11T08:34:16.902 INFO:journalctl@ceph.mon.c.smithi131.stdout: partition_filters: 0 2024-09-11T08:34:16.902 INFO:journalctl@ceph.mon.c.smithi131.stdout: use_delta_encoding: 1 2024-09-11T08:34:16.903 INFO:journalctl@ceph.mon.c.smithi131.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T08:34:16.903 INFO:journalctl@ceph.mon.c.smithi131.stdout: whole_key_filtering: 1 2024-09-11T08:34:16.903 INFO:journalctl@ceph.mon.c.smithi131.stdout: verify_compression: 0 2024-09-11T08:34:16.903 INFO:journalctl@ceph.mon.c.smithi131.stdout: read_amp_bytes_per_bit: 0 2024-09-11T08:34:16.903 INFO:journalctl@ceph.mon.c.smithi131.stdout: format_version: 4 2024-09-11T08:34:16.903 INFO:journalctl@ceph.mon.c.smithi131.stdout: enable_index_compression: 1 2024-09-11T08:34:16.903 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_align: 0 2024-09-11T08:34:16.903 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T08:34:16.903 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T08:34:16.903 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compression: NoCompression 2024-09-11T08:34:16.903 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T08:34:16.903 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T08:34:16.903 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T08:34:16.903 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.num_levels: 7 2024-09-11T08:34:16.903 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T08:34:16.903 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T08:34:16.904 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T08:34:16.904 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T08:34:16.904 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T08:34:16.904 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T08:34:16.904 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T08:34:16.904 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:34:16.904 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T08:34:16.904 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T08:34:16.904 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T08:34:16.904 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T08:34:16.904 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T08:34:16.904 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T08:34:16.904 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T08:34:16.904 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T08:34:16.904 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T08:34:16.905 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T08:34:16.905 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T08:34:16.905 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T08:34:16.905 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T08:34:16.905 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T08:34:16.905 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T08:34:16.905 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T08:34:16.905 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T08:34:16.905 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T08:34:16.905 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T08:34:16.905 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T08:34:16.905 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T08:34:16.905 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T08:34:16.905 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T08:34:16.906 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T08:34:16.906 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T08:34:16.906 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T08:34:16.906 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T08:34:16.906 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T08:34:16.906 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T08:34:16.906 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T08:34:16.906 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T08:34:16.906 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T08:34:16.906 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T08:34:16.906 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T08:34:16.906 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T08:34:16.906 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T08:34:16.906 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T08:34:16.907 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T08:34:16.907 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T08:34:16.907 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T08:34:16.907 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T08:34:16.907 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T08:34:16.907 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.inplace_update_support: 0 2024-09-11T08:34:16.907 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T08:34:16.907 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T08:34:16.907 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T08:34:16.907 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T08:34:16.907 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.bloom_locality: 0 2024-09-11T08:34:16.907 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.max_successive_merges: 0 2024-09-11T08:34:16.907 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T08:34:16.907 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T08:34:16.907 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T08:34:16.908 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T08:34:16.908 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.ttl: 2592000 2024-09-11T08:34:16.908 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T08:34:16.908 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.enable_blob_files: false 2024-09-11T08:34:16.908 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.min_blob_size: 0 2024-09-11T08:34:16.908 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T08:34:16.908 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T08:34:16.908 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T08:34:16.908 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T08:34:16.908 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T08:34:16.908 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-11T08:34:16.908 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-11T08:34:16.908 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043656455737, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-11T08:34:16.908 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-11T08:34:16.909 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T08:34:16.909 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043656456043, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1626, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 578, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 456, "raw_average_value_size": 91, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726043656, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "a9fdcfe0-73b1-412c-9097-eb8024c21659", "db_session_id": "SPSCLO8XZBUAIQGLLKC8"}} 2024-09-11T08:34:16.909 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-11T08:34:16.909 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726043656456357, "job": 1, "event": "recovery_finished"} 2024-09-11T08:34:16.909 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T08:34:16.909 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5652cb3e2700 2024-09-11T08:34:16.909 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: DB pointer 0x5652cb3b2000 2024-09-11T08:34:16.909 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T08:34:16.909 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T08:34:16.909 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** DB Stats ** 2024-09-11T08:34:16.909 INFO:journalctl@ceph.mon.c.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:34:16.909 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T08:34:16.909 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T08:34:16.910 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:34:16.910 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T08:34:16.910 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T08:34:16.910 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T08:34:16.910 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-11T08:34:16.910 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-11T08:34:16.910 INFO:journalctl@ceph.mon.c.smithi131.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:34:16.910 INFO:journalctl@ceph.mon.c.smithi131.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:34:16.910 INFO:journalctl@ceph.mon.c.smithi131.stdout: L0 1/0 1.59 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-11T08:34:16.910 INFO:journalctl@ceph.mon.c.smithi131.stdout: Sum 1/0 1.59 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-11T08:34:16.910 INFO:journalctl@ceph.mon.c.smithi131.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-11T08:34:16.910 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-11T08:34:16.910 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-11T08:34:16.910 INFO:journalctl@ceph.mon.c.smithi131.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:34:16.910 INFO:journalctl@ceph.mon.c.smithi131.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:34:16.911 INFO:journalctl@ceph.mon.c.smithi131.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-11T08:34:16.911 INFO:journalctl@ceph.mon.c.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:34:16.911 INFO:journalctl@ceph.mon.c.smithi131.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:34:16.911 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:34:16.911 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:34:16.911 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:34:16.911 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:34:16.911 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:34:16.911 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:34:16.911 INFO:journalctl@ceph.mon.c.smithi131.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:34:16.911 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-11T08:34:16.911 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:34:16.911 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-11T08:34:16.912 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-11T08:34:16.912 INFO:journalctl@ceph.mon.c.smithi131.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:34:16.912 INFO:journalctl@ceph.mon.c.smithi131.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:34:16.912 INFO:journalctl@ceph.mon.c.smithi131.stdout: L0 1/0 1.59 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-11T08:34:16.912 INFO:journalctl@ceph.mon.c.smithi131.stdout: Sum 1/0 1.59 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-11T08:34:16.912 INFO:journalctl@ceph.mon.c.smithi131.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T08:34:16.912 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-11T08:34:16.912 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-11T08:34:16.912 INFO:journalctl@ceph.mon.c.smithi131.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T08:34:16.912 INFO:journalctl@ceph.mon.c.smithi131.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T08:34:16.912 INFO:journalctl@ceph.mon.c.smithi131.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-11T08:34:16.912 INFO:journalctl@ceph.mon.c.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T08:34:16.912 INFO:journalctl@ceph.mon.c.smithi131.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T08:34:16.913 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T08:34:16.913 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T08:34:16.913 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T08:34:16.913 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T08:34:16.913 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:34:16.913 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T08:34:16.913 INFO:journalctl@ceph.mon.c.smithi131.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T08:34:16.913 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-11T08:34:16.913 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T08:34:16.913 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-11T08:34:16.913 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: using public_addrv v1:172.21.15.131:6790/0 2024-09-11T08:34:16.913 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: starting mon.c rank -1 at public addrs v1:172.21.15.131:6790/0 at bind addrs v1:172.21.15.131:6790/0 mon_data /var/lib/ceph/mon/ceph-c fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:34:16.913 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.c@-1(???) e0 preinit fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:34:16.913 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.c@-1(synchronizing).mds e1 new map 2024-09-11T08:34:16.913 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-11T08:34:16.914 INFO:journalctl@ceph.mon.c.smithi131.stdout: e1 2024-09-11T08:34:16.914 INFO:journalctl@ceph.mon.c.smithi131.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T08:34:16.914 INFO:journalctl@ceph.mon.c.smithi131.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T08:34:16.914 INFO:journalctl@ceph.mon.c.smithi131.stdout: legacy client fscid: -1 2024-09-11T08:34:16.914 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-11T08:34:16.914 INFO:journalctl@ceph.mon.c.smithi131.stdout: No filesystems configured 2024-09-11T08:34:16.914 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-11T08:34:16.914 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-11T08:34:16.914 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-11T08:34:16.914 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-11T08:34:16.914 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-11T08:34:16.914 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-11T08:34:16.914 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T08:34:16.914 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:34:16.915 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:34:16.915 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T08:34:16.915 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:16.915 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='client.14182 v1:172.21.15.189:0/1894932059' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:[v1:172.21.15.131:6789]=a;smithi131:[v1:172.21.15.131:6790]=c;smithi189:[v1:172.21.15.189:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:16.915 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: Saving service mon spec with placement smithi131:[v1:172.21.15.131:6789]=a;smithi131:[v1:172.21.15.131:6790]=c;smithi189:[v1:172.21.15.189:6789]=b;count:3 2024-09-11T08:34:16.915 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:16.915 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:16.915 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:16.915 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:16.915 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:16.915 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:16.915 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:16.915 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: Deploying daemon mon.b on smithi189 2024-09-11T08:34:16.915 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:16.915 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:34:16.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:16.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.a calling monitor election 2024-09-11T08:34:16.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:16.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:16.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:16.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.b calling monitor election 2024-09-11T08:34:16.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:16.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:16.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:16.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:16.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T08:34:16.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: monmap e2: 2 mons at {a=v1:172.21.15.131:6789/0,b=v1:172.21.15.189:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:34:16.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: fsmap 2024-09-11T08:34:16.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:34:16.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mgrmap e13: y(active, since 37s) 2024-09-11T08:34:16.917 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: overall HEALTH_OK 2024-09-11T08:34:16.917 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:16.917 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:16.917 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:16.917 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:16.917 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:16.917 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: Deploying daemon mon.c on smithi131 2024-09-11T08:34:16.917 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:16.917 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.189:0/4007520027' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:34:16.917 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:16.917 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 ceph-mon[37208]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-11T08:34:16.917 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 bash[37179]: 569d21b6765f851a50830410f022e15ec8da7ff7ab30888d1f4577303fc07168 2024-09-11T08:34:16.917 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:16 smithi131 systemd[1]: Started Ceph mon.c for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:34:21.741 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-11T08:34:21.742 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":3,"fsid":"50e4d0bc-7018-11ef-bcea-c7b262605968","modified":"2024-09-11T08:34:16.475916Z","created":"2024-09-11T08:32:49.512160Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6790","nonce":0}]},"addr":"172.21.15.131:6790/0","public_addr":"172.21.15.131:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-11T08:34:21.742 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 3 2024-09-11T08:34:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:34:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: mon.a calling monitor election 2024-09-11T08:34:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: mon.b calling monitor election 2024-09-11T08:34:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: mon.c calling monitor election 2024-09-11T08:34:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T08:34:21.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: monmap e3: 3 mons at {a=v1:172.21.15.131:6789/0,b=v1:172.21.15.189:6789/0,c=v1:172.21.15.131:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:34:21.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: fsmap 2024-09-11T08:34:21.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:34:21.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: mgrmap e13: y(active, since 46s) 2024-09-11T08:34:21.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: overall HEALTH_OK 2024-09-11T08:34:21.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:21.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:21.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:21.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:21.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:34:21.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: mon.a calling monitor election 2024-09-11T08:34:21.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:21.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: mon.b calling monitor election 2024-09-11T08:34:21.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:21.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: mon.c calling monitor election 2024-09-11T08:34:21.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:21.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T08:34:21.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: monmap e3: 3 mons at {a=v1:172.21.15.131:6789/0,b=v1:172.21.15.189:6789/0,c=v1:172.21.15.131:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:34:21.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: fsmap 2024-09-11T08:34:21.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:34:21.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: mgrmap e13: y(active, since 46s) 2024-09-11T08:34:21.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: overall HEALTH_OK 2024-09-11T08:34:21.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:21.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:21.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:21.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:21 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:21.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:34:21.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: mon.a calling monitor election 2024-09-11T08:34:21.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:34:21.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: mon.b calling monitor election 2024-09-11T08:34:21.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:21.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: mon.c calling monitor election 2024-09-11T08:34:21.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:21.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:21.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T08:34:21.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: monmap e3: 3 mons at {a=v1:172.21.15.131:6789/0,b=v1:172.21.15.189:6789/0,c=v1:172.21.15.131:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T08:34:21.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: fsmap 2024-09-11T08:34:21.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T08:34:21.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: mgrmap e13: y(active, since 46s) 2024-09-11T08:34:21.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: overall HEALTH_OK 2024-09-11T08:34:21.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:21.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:21.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:21.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:21 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:22.317 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T08:34:22.317 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T08:34:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:34:22.670 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:22 smithi189 ceph-mon[32675]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:22.671 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:22 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:22.671 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:22 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.189:0/755767812' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:34:22.671 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:22 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:22 smithi131 ceph-mon[29446]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:22 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:22 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.189:0/755767812' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:34:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:22 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:22.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:22 smithi131 ceph-mon[37208]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:22.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:22 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:22.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:22 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.189:0/755767812' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T08:34:22.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:22 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:34:23.783 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:34:23.475+0000 7ffbd9999640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-11T08:34:23.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:23.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:23.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[37208]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-11T08:34:23.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[37208]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-11T08:34:23.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:23.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:34:23.784 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:23.785 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.785 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.785 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.785 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.785 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:23.785 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:23.785 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[29446]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-11T08:34:23.785 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[29446]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-11T08:34:23.785 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.785 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.785 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.785 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.786 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.786 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:23.786 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:34:23.786 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:23.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:23.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:23.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:23 smithi189 ceph-mon[32675]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-11T08:34:23.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:23 smithi189 ceph-mon[32675]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-11T08:34:23.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:23.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:23.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:34:23.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:24.600 INFO:teuthology.orchestra.run.smithi131.stdout:# minimal ceph.conf for 50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:34:24.601 INFO:teuthology.orchestra.run.smithi131.stdout:[global] 2024-09-11T08:34:24.601 INFO:teuthology.orchestra.run.smithi131.stdout: fsid = 50e4d0bc-7018-11ef-bcea-c7b262605968 2024-09-11T08:34:24.601 INFO:teuthology.orchestra.run.smithi131.stdout: mon_host = 172.21.15.131:6789/0 172.21.15.189:6789/0 v1:172.21.15.131:6790/0 2024-09-11T08:34:25.341 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:25 smithi131 ceph-mon[29446]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:34:25.342 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:25 smithi131 ceph-mon[29446]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:34:25.342 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:25 smithi131 ceph-mon[29446]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:25.342 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:25 smithi131 ceph-mon[29446]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T08:34:25.342 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:25 smithi131 ceph-mon[29446]: Reconfiguring daemon mon.a on smithi131 2024-09-11T08:34:25.342 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:25 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1562359873' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:25.342 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:25 smithi131 ceph-mon[37208]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:34:25.343 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:25 smithi131 ceph-mon[37208]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:34:25.343 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:25 smithi131 ceph-mon[37208]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:25.343 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:25 smithi131 ceph-mon[37208]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T08:34:25.343 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:25 smithi131 ceph-mon[37208]: Reconfiguring daemon mon.a on smithi131 2024-09-11T08:34:25.343 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:25 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1562359873' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:25.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:25 smithi189 ceph-mon[32675]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:34:25.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:25 smithi189 ceph-mon[32675]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:34:25.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:25 smithi189 ceph-mon[32675]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:25.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:25 smithi189 ceph-mon[32675]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T08:34:25.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:25 smithi189 ceph-mon[32675]: Reconfiguring daemon mon.a on smithi131 2024-09-11T08:34:25.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:25 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1562359873' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:25.962 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T08:34:25.962 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-11T08:34:25.962 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T08:34:25.990 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-11T08:34:25.990 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:34:26.055 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-11T08:34:26.055 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T08:34:26.088 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-11T08:34:26.089 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:34:26.159 INFO:tasks.cephadm:Adding mgr.y on smithi131 2024-09-11T08:34:26.159 INFO:tasks.cephadm:Adding mgr.x on smithi189 2024-09-11T08:34:26.159 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi131=y;smithi189=x' 2024-09-11T08:34:26.444 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:34:26.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:26 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:26.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:26 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:26.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:26 smithi189 ceph-mon[32675]: Reconfiguring mon.c (monmap changed)... 2024-09-11T08:34:26.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:26 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:26.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:26 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:34:26.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:26 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:26.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:26 smithi189 ceph-mon[32675]: Reconfiguring daemon mon.c on smithi131 2024-09-11T08:34:26.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:26 smithi189 ceph-mon[32675]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:26.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:26 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:26.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:26.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:26.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[29446]: Reconfiguring mon.c (monmap changed)... 2024-09-11T08:34:26.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:26.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:34:26.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:26.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[29446]: Reconfiguring daemon mon.c on smithi131 2024-09-11T08:34:26.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[29446]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:26.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:26.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:26.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:26.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[37208]: Reconfiguring mon.c (monmap changed)... 2024-09-11T08:34:26.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:26.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:34:26.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:26.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[37208]: Reconfiguring daemon mon.c on smithi131 2024-09-11T08:34:26.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[37208]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:26.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:26 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:28.206 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mgr update... 2024-09-11T08:34:28.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:28 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:28.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:28 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:28.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:28 smithi131 ceph-mon[29446]: Reconfiguring mon.b (monmap changed)... 2024-09-11T08:34:28.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:28 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:28.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:28 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:34:28.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:28 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:28.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:28 smithi131 ceph-mon[29446]: Reconfiguring daemon mon.b on smithi189 2024-09-11T08:34:28.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:28 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:28.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:28 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:28.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:28 smithi131 ceph-mon[37208]: Reconfiguring mon.b (monmap changed)... 2024-09-11T08:34:28.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:28 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:28.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:28 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:34:28.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:28 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:28.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:28 smithi131 ceph-mon[37208]: Reconfiguring daemon mon.b on smithi189 2024-09-11T08:34:28.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:28 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:28.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:28 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:28.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:28 smithi189 ceph-mon[32675]: Reconfiguring mon.b (monmap changed)... 2024-09-11T08:34:28.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:28 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T08:34:28.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:28 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T08:34:28.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:28 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:28.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:28 smithi189 ceph-mon[32675]: Reconfiguring daemon mon.b on smithi189 2024-09-11T08:34:28.869 DEBUG:teuthology.orchestra.run.smithi189:mgr.x> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T08:34:28.871 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T08:34:28.871 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-11T08:34:28.871 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T08:34:28.891 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T08:34:28.891 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2024-09-11T08:34:28.950 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T08:34:28.950 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:34:28.950 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 969 Links: 1 2024-09-11T08:34:28.950 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:34:28.950 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:34:28.950 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-11 08:34:15.842739104 +0000 2024-09-11T08:34:28.950 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-11 08:29:45.594472438 +0000 2024-09-11T08:34:28.950 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-11 08:29:45.594472438 +0000 2024-09-11T08:34:28.950 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-09-11 08:29:45.594472438 +0000 2024-09-11T08:34:28.951 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T08:34:29.018 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-11T08:34:29.018 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-11T08:34:29.018 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000248285 s, 2.1 MB/s 2024-09-11T08:34:29.020 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T08:34:29.078 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2024-09-11T08:34:29.136 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T08:34:29.136 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:34:29.136 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 982 Links: 1 2024-09-11T08:34:29.136 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:34:29.136 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:34:29.136 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-11 08:34:15.843739083 +0000 2024-09-11T08:34:29.136 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-11 08:29:45.597472374 +0000 2024-09-11T08:34:29.136 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-11 08:29:45.597472374 +0000 2024-09-11T08:34:29.137 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-09-11 08:29:45.597472374 +0000 2024-09-11T08:34:29.137 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T08:34:29.202 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-11T08:34:29.202 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-11T08:34:29.202 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000231435 s, 2.2 MB/s 2024-09-11T08:34:29.203 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T08:34:29.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:29 smithi189 ceph-mon[32675]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:29.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:29 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:29.261 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2024-09-11T08:34:29.292 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:29 smithi131 ceph-mon[37208]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:29.292 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:29 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:29.293 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:29 smithi131 ceph-mon[29446]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:29.293 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:29 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:29.309 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T08:34:29.310 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:34:29.310 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 977 Links: 1 2024-09-11T08:34:29.310 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:34:29.310 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:34:29.310 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-11 08:34:15.843739083 +0000 2024-09-11T08:34:29.310 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-11 08:29:45.595472417 +0000 2024-09-11T08:34:29.311 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-11 08:29:45.595472417 +0000 2024-09-11T08:34:29.311 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-09-11 08:29:45.595472417 +0000 2024-09-11T08:34:29.311 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T08:34:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-11T08:34:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-11T08:34:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000203074 s, 2.5 MB/s 2024-09-11T08:34:29.377 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T08:34:29.434 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2024-09-11T08:34:29.490 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T08:34:29.491 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:34:29.491 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-11T08:34:29.491 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:34:29.491 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:34:29.491 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-11 08:34:15.844739062 +0000 2024-09-11T08:34:29.491 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-11 08:29:45.595472417 +0000 2024-09-11T08:34:29.491 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-11 08:29:45.595472417 +0000 2024-09-11T08:34:29.491 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-09-11 08:29:45.595472417 +0000 2024-09-11T08:34:29.491 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T08:34:29.555 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-11T08:34:29.555 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-11T08:34:29.555 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000219927 s, 2.3 MB/s 2024-09-11T08:34:29.557 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T08:34:29.613 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-11T08:34:29.614 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T08:34:29.630 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T08:34:29.631 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 2024-09-11T08:34:29.687 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T08:34:29.687 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:34:29.687 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 982 Links: 1 2024-09-11T08:34:29.688 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:34:29.688 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:34:29.688 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-11 08:34:07.342697325 +0000 2024-09-11T08:34:29.688 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-11 08:29:45.545669487 +0000 2024-09-11T08:34:29.688 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-11 08:29:45.545669487 +0000 2024-09-11T08:34:29.688 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-11 08:29:45.545669487 +0000 2024-09-11T08:34:29.689 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T08:34:29.756 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-11T08:34:29.756 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-11T08:34:29.756 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000243491 s, 2.1 MB/s 2024-09-11T08:34:29.757 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T08:34:29.813 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 2024-09-11T08:34:29.869 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T08:34:29.870 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:34:29.870 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 969 Links: 1 2024-09-11T08:34:29.870 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:34:29.870 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:34:29.870 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-11 08:34:07.343697306 +0000 2024-09-11T08:34:29.870 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-11 08:29:45.542669545 +0000 2024-09-11T08:34:29.870 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-11 08:29:45.542669545 +0000 2024-09-11T08:34:29.870 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-11 08:29:45.542669545 +0000 2024-09-11T08:34:29.870 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T08:34:29.935 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-11T08:34:29.935 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-11T08:34:29.935 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000194678 s, 2.6 MB/s 2024-09-11T08:34:29.936 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T08:34:29.994 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 2024-09-11T08:34:30.050 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T08:34:30.050 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:34:30.050 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-11T08:34:30.050 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:34:30.050 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:34:30.050 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-11 08:34:07.343697306 +0000 2024-09-11T08:34:30.050 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-11 08:29:45.543669526 +0000 2024-09-11T08:34:30.050 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-11 08:29:45.543669526 +0000 2024-09-11T08:34:30.050 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-11 08:29:45.543669526 +0000 2024-09-11T08:34:30.051 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T08:34:30.114 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-11T08:34:30.114 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-11T08:34:30.114 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000234853 s, 2.2 MB/s 2024-09-11T08:34:30.115 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T08:34:30.173 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 2024-09-11T08:34:30.231 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T08:34:30.231 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T08:34:30.231 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 977 Links: 1 2024-09-11T08:34:30.231 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T08:34:30.231 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T08:34:30.231 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-11 08:34:07.343697306 +0000 2024-09-11T08:34:30.231 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-11 08:29:45.544669506 +0000 2024-09-11T08:34:30.232 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-11 08:29:45.544669506 +0000 2024-09-11T08:34:30.232 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-11 08:29:45.544669506 +0000 2024-09-11T08:34:30.232 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T08:34:30.297 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-11T08:34:30.297 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-11T08:34:30.297 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000209727 s, 2.4 MB/s 2024-09-11T08:34:30.299 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T08:34:30.355 INFO:tasks.cephadm:Deploying osd.0 on smithi131 with /dev/vg_nvme/lv_4... 2024-09-11T08:34:30.356 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-11T08:34:30.370 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:30 smithi189 ceph-mon[32675]: from='client.24103 v1:172.21.15.189:0/3639337540' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi131=y;smithi189=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:30.370 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:30 smithi189 ceph-mon[32675]: Saving service mgr spec with placement smithi131=y;smithi189=x;count:2 2024-09-11T08:34:30.370 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:30 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:30.370 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:30 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:30.370 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:30 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:30.370 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:30 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:30.370 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:30 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:30.370 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:30 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:30.370 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:30 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:34:30.370 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:30 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T08:34:30.370 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:30 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:34:30.370 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:30 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:30.378 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[37208]: from='client.24103 v1:172.21.15.189:0/3639337540' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi131=y;smithi189=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:30.378 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[37208]: Saving service mgr spec with placement smithi131=y;smithi189=x;count:2 2024-09-11T08:34:30.378 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:30.378 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:30.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:30.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:30.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:30.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:30.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:34:30.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T08:34:30.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:34:30.379 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:30.379 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[29446]: from='client.24103 v1:172.21.15.189:0/3639337540' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi131=y;smithi189=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:30.379 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[29446]: Saving service mgr spec with placement smithi131=y;smithi189=x;count:2 2024-09-11T08:34:30.379 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:30.379 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:30.380 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:30.380 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:30.380 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:30.380 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:30.380 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:34:30.380 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T08:34:30.380 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:34:30.380 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:30 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:30.601 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:34:31.387 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:31 smithi131 ceph-mon[37208]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:31.387 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:31 smithi131 ceph-mon[37208]: Deploying daemon mgr.x on smithi189 2024-09-11T08:34:31.387 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:31 smithi131 ceph-mon[29446]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:31.387 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:31 smithi131 ceph-mon[29446]: Deploying daemon mgr.x on smithi189 2024-09-11T08:34:31.469 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:31 smithi189 ceph-mon[32675]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:31.470 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:31 smithi189 ceph-mon[32675]: Deploying daemon mgr.x on smithi189 2024-09-11T08:34:31.470 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:31 smithi189 systemd[1]: Starting Ceph mgr.x for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:34:31.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:31 smithi189 podman[34490]: 2024-09-11 08:34:31.683784112 +0000 UTC m=+0.112287285 container create 9a2cf3ca6884231bfca591d5e8604c6d0268882b2a61fa8903741ff46a530895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:34:31.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:31 smithi189 podman[34490]: 2024-09-11 08:34:31.587033343 +0000 UTC m=+0.015536525 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:34:32.037 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:31 smithi189 podman[34490]: 2024-09-11 08:34:31.861159823 +0000 UTC m=+0.289663006 container init 9a2cf3ca6884231bfca591d5e8604c6d0268882b2a61fa8903741ff46a530895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:34:32.037 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:31 smithi189 podman[34490]: 2024-09-11 08:34:31.863848984 +0000 UTC m=+0.292352158 container start 9a2cf3ca6884231bfca591d5e8604c6d0268882b2a61fa8903741ff46a530895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T08:34:32.037 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:31 smithi189 bash[34490]: 9a2cf3ca6884231bfca591d5e8604c6d0268882b2a61fa8903741ff46a530895 2024-09-11T08:34:32.037 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:31 smithi189 systemd[1]: Started Ceph mgr.x for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:34:32.038 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:31 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:31.978+0000 7fdd26391ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:34:32.038 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:32 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:32.036+0000 7fdd26391ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:34:32.278 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-11T08:34:32.364 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-11T08:34:32.379 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-09-11T08:34:32.379 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-09-11T08:34:32.379 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133532 s, 785 MB/s 2024-09-11T08:34:32.379 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-09-11T08:34:32.384 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:32 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:32.103+0000 7fdd26391ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:34:32.384 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:32 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:32.234+0000 7fdd26391ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:34:32.384 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:32 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:32.383+0000 7fdd26391ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:34:32.919 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_4 2024-09-11T08:34:32.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:32 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:32.654+0000 7fdd26391ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:34:32.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:32 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:32.717+0000 7fdd26391ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:34:32.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:32 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:32.884+0000 7fdd26391ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:34:33.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:33 smithi131 ceph-mon[29446]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:33.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:33 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:33.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:33 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:33.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:33 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:33.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:33 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:33.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:33 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:33.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:33 smithi131 ceph-mon[37208]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:33.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:33 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:33.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:33 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:33.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:33 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:33.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:33 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:33.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:33 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:33.205 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:34:33.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:33 smithi189 ceph-mon[32675]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:33.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:33 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:33.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:33 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:33.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:33 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:33.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:33 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:33.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:33 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:34.163 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:33 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:33.743+0000 7fdd26391ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:34:34.163 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:33 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:33.807+0000 7fdd26391ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:34:34.312 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:34.312 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:34.312 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:34.312 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:34.312 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:34.312 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[29446]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T08:34:34.312 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:34:34.312 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:34:34.312 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:34.312 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[29446]: Reconfiguring daemon mgr.y on smithi131 2024-09-11T08:34:34.313 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[29446]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:34.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:34.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:34.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:34.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:34.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:34.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[37208]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T08:34:34.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:34:34.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:34:34.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:34.314 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[37208]: Reconfiguring daemon mgr.y on smithi131 2024-09-11T08:34:34.314 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:34 smithi131 ceph-mon[37208]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:34.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:34.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:34.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:34.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:34.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:34.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-mon[32675]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T08:34:34.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:34:34.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:34:34.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:34.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-mon[32675]: Reconfiguring daemon mgr.y on smithi131 2024-09-11T08:34:34.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-mon[32675]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:34.802 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:34.517+0000 7fdd26391ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:34:34.803 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:34.626+0000 7fdd26391ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:34:34.803 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:34.744+0000 7fdd26391ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:34:34.803 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:34.802+0000 7fdd26391ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:34:35.183 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:34.927+0000 7fdd26391ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:34:35.183 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:34 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:34.994+0000 7fdd26391ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:34:35.183 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:35 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:35.051+0000 7fdd26391ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:34:35.183 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:35 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:35.183+0000 7fdd26391ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:34:35.445 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:35 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:35.246+0000 7fdd26391ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:34:35.445 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:35 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:35.309+0000 7fdd26391ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:34:35.445 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:35 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:35.388+0000 7fdd26391ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:34:35.445 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:35 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:35.445+0000 7fdd26391ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:34:35.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:35 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:35.502+0000 7fdd26391ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:34:36.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:36.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:36.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: from='client.14214 v1:172.21.15.131:0/3010857668' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:36.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:34:36.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:34:36.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:36.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:36.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:36.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:36.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:36.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:36.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:36.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: Standby manager daemon x started 2024-09-11T08:34:36.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: from='mgr.? v1:172.21.15.189:0/1749774612' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:34:36.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: from='mgr.? v1:172.21.15.189:0/1749774612' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:34:36.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: from='mgr.? v1:172.21.15.189:0/1749774612' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:34:36.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[37208]: from='mgr.? v1:172.21.15.189:0/1749774612' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:34:36.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:36.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:36.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: from='client.14214 v1:172.21.15.131:0/3010857668' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:36.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:34:36.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:34:36.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:36.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:36.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:36.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:36.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:36.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:36.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:36.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: Standby manager daemon x started 2024-09-11T08:34:36.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: from='mgr.? v1:172.21.15.189:0/1749774612' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:34:36.643 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: from='mgr.? v1:172.21.15.189:0/1749774612' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:34:36.644 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: from='mgr.? v1:172.21.15.189:0/1749774612' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:34:36.644 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:36 smithi131 ceph-mon[29446]: from='mgr.? v1:172.21.15.189:0/1749774612' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:34:36.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:34:36.242+0000 7fdd26391ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:34:36.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:36.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:36.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: from='client.14214 v1:172.21.15.131:0/3010857668' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:34:36.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:34:36.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:34:36.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:36.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:36.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:36.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:36.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:36.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:36.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:36.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: Standby manager daemon x started 2024-09-11T08:34:36.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: from='mgr.? v1:172.21.15.189:0/1749774612' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:34:36.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: from='mgr.? v1:172.21.15.189:0/1749774612' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:34:36.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: from='mgr.? v1:172.21.15.189:0/1749774612' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:34:36.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:36 smithi189 ceph-mon[32675]: from='mgr.? v1:172.21.15.189:0/1749774612' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:34:38.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:38 smithi131 ceph-mon[29446]: mgrmap e14: y(active, since 61s), standbys: x 2024-09-11T08:34:38.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:38 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:34:38.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:38 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/118976584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a4b0f7e-5c59-472b-ac6b-fefc3f09e091"}]: dispatch 2024-09-11T08:34:38.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:38 smithi131 ceph-mon[29446]: from='client.24119 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a4b0f7e-5c59-472b-ac6b-fefc3f09e091"}]: dispatch 2024-09-11T08:34:38.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:38 smithi131 ceph-mon[37208]: mgrmap e14: y(active, since 61s), standbys: x 2024-09-11T08:34:38.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:38 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:34:38.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:38 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/118976584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a4b0f7e-5c59-472b-ac6b-fefc3f09e091"}]: dispatch 2024-09-11T08:34:38.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:38 smithi131 ceph-mon[37208]: from='client.24119 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a4b0f7e-5c59-472b-ac6b-fefc3f09e091"}]: dispatch 2024-09-11T08:34:38.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:38 smithi189 ceph-mon[32675]: mgrmap e14: y(active, since 61s), standbys: x 2024-09-11T08:34:38.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:38 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:34:38.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:38 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/118976584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a4b0f7e-5c59-472b-ac6b-fefc3f09e091"}]: dispatch 2024-09-11T08:34:38.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:38 smithi189 ceph-mon[32675]: from='client.24119 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a4b0f7e-5c59-472b-ac6b-fefc3f09e091"}]: dispatch 2024-09-11T08:34:39.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:39 smithi131 ceph-mon[37208]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:39.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:39 smithi131 ceph-mon[37208]: from='client.24119 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a4b0f7e-5c59-472b-ac6b-fefc3f09e091"}]': finished 2024-09-11T08:34:39.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:39 smithi131 ceph-mon[37208]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T08:34:39.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:39 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:34:39.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:39 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1791586444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:34:39.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:39 smithi131 ceph-mon[29446]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:39.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:39 smithi131 ceph-mon[29446]: from='client.24119 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a4b0f7e-5c59-472b-ac6b-fefc3f09e091"}]': finished 2024-09-11T08:34:39.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:39 smithi131 ceph-mon[29446]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T08:34:39.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:39 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:34:39.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:39 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1791586444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:34:39.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:39 smithi189 ceph-mon[32675]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:39.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:39 smithi189 ceph-mon[32675]: from='client.24119 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a4b0f7e-5c59-472b-ac6b-fefc3f09e091"}]': finished 2024-09-11T08:34:39.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:39 smithi189 ceph-mon[32675]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T08:34:39.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:39 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:34:39.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:39 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1791586444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:34:41.321 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:41 smithi131 ceph-mon[29446]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:41.321 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:41 smithi131 ceph-mon[37208]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:41.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:41 smithi189 ceph-mon[32675]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:43.266 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:43 smithi131 ceph-mon[29446]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:43.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:43 smithi131 ceph-mon[37208]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:43.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:43 smithi189 ceph-mon[32675]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:44.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:44 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T08:34:44.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:44 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:44.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:44 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T08:34:44.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:44 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:44.391 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:44 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T08:34:44.392 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:44 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:45.340 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:45 smithi131 ceph-mon[29446]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:45.340 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:45 smithi131 ceph-mon[29446]: Deploying daemon osd.0 on smithi131 2024-09-11T08:34:45.341 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:45 smithi131 ceph-mon[37208]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:45.341 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:45 smithi131 ceph-mon[37208]: Deploying daemon osd.0 on smithi131 2024-09-11T08:34:45.406 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:45 smithi189 ceph-mon[32675]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:45.406 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:45 smithi189 ceph-mon[32675]: Deploying daemon osd.0 on smithi131 2024-09-11T08:34:47.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:47 smithi131 ceph-mon[29446]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:47.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:47 smithi131 ceph-mon[37208]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:47.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:47 smithi189 ceph-mon[32675]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:49.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:49 smithi131 ceph-mon[29446]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:49.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:49.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:49.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:49.309 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:49 smithi131 ceph-mon[37208]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:49.309 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:49.309 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:49.309 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:49.501 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:49 smithi189 ceph-mon[32675]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:49.501 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:49.502 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:49.502 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:51.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:51 smithi131 ceph-mon[29446]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:51.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:51 smithi131 ceph-mon[37208]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:51.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:51 smithi189 ceph-mon[32675]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:52.117 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 0 on host 'smithi131' 2024-09-11T08:34:52.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:52.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:52.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[29446]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:52.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:52.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:52.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:52.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[29446]: from='osd.0 v1:172.21.15.131:6801/1075394441' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:34:52.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[29446]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:34:52.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:52.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:52.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:52.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:52.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:52.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[37208]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:52.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:52.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:52.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:52.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[37208]: from='osd.0 v1:172.21.15.131:6801/1075394441' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:34:52.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[37208]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:34:52.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:52.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:52.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:52 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:52.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:52 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:52.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:52 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:52.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:52 smithi189 ceph-mon[32675]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:52.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:52 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:52.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:52 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:52.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:52 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:52.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:52 smithi189 ceph-mon[32675]: from='osd.0 v1:172.21.15.131:6801/1075394441' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:34:52.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:52 smithi189 ceph-mon[32675]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T08:34:52.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:52 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:34:52.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:52 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:52.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:52 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:52.861 DEBUG:teuthology.orchestra.run.smithi131:osd.0> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T08:34:52.863 INFO:tasks.cephadm:Deploying osd.1 on smithi131 with /dev/vg_nvme/lv_3... 2024-09-11T08:34:52.863 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-11T08:34:54.011 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:53 smithi131 ceph-mon[29446]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T08:34:54.011 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:53 smithi131 ceph-mon[29446]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T08:34:54.011 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:53 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:34:54.011 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:53 smithi131 ceph-mon[29446]: from='osd.0 v1:172.21.15.131:6801/1075394441' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:34:54.012 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:53 smithi131 ceph-mon[29446]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:34:54.012 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:53 smithi131 ceph-mon[37208]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T08:34:54.012 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:53 smithi131 ceph-mon[37208]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T08:34:54.012 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:53 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:34:54.012 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:53 smithi131 ceph-mon[37208]: from='osd.0 v1:172.21.15.131:6801/1075394441' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:34:54.012 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:53 smithi131 ceph-mon[37208]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:34:54.012 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:34:53 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-0[41071]: 2024-09-11T08:34:53.825+0000 7f123c0e9640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T08:34:54.013 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:34:53 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-0[41071]: 2024-09-11T08:34:53.827+0000 7f1236f20640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:34:54.046 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:34:54.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:53 smithi189 ceph-mon[32675]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T08:34:54.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:53 smithi189 ceph-mon[32675]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T08:34:54.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:53 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:34:54.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:53 smithi189 ceph-mon[32675]: from='osd.0 v1:172.21.15.131:6801/1075394441' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:34:54.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:53 smithi189 ceph-mon[32675]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:34:55.066 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:54 smithi131 ceph-mon[29446]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:55.066 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:54 smithi131 ceph-mon[29446]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-11T08:34:55.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:54 smithi131 ceph-mon[29446]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T08:34:55.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:54 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:34:55.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:54 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:34:55.067 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:54 smithi131 ceph-mon[37208]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:55.067 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:54 smithi131 ceph-mon[37208]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-11T08:34:55.067 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:54 smithi131 ceph-mon[37208]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T08:34:55.067 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:54 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:34:55.067 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:54 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:34:55.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:54 smithi189 ceph-mon[32675]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:55.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:54 smithi189 ceph-mon[32675]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-11T08:34:55.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:54 smithi189 ceph-mon[32675]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T08:34:55.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:54 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:34:55.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:54 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:34:56.119 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:55 smithi189 ceph-mon[32675]: purged_snaps scrub starts 2024-09-11T08:34:56.119 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:55 smithi189 ceph-mon[32675]: purged_snaps scrub ok 2024-09-11T08:34:56.119 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:55 smithi189 ceph-mon[32675]: osd.0 v1:172.21.15.131:6801/1075394441 boot 2024-09-11T08:34:56.119 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:55 smithi189 ceph-mon[32675]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T08:34:56.119 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:55 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:34:56.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:55 smithi131 ceph-mon[37208]: purged_snaps scrub starts 2024-09-11T08:34:56.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:55 smithi131 ceph-mon[37208]: purged_snaps scrub ok 2024-09-11T08:34:56.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:55 smithi131 ceph-mon[37208]: osd.0 v1:172.21.15.131:6801/1075394441 boot 2024-09-11T08:34:56.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:55 smithi131 ceph-mon[37208]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T08:34:56.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:55 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:34:56.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:55 smithi131 ceph-mon[29446]: purged_snaps scrub starts 2024-09-11T08:34:56.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:55 smithi131 ceph-mon[29446]: purged_snaps scrub ok 2024-09-11T08:34:56.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:55 smithi131 ceph-mon[29446]: osd.0 v1:172.21.15.131:6801/1075394441 boot 2024-09-11T08:34:56.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:55 smithi131 ceph-mon[29446]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T08:34:56.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:55 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:34:56.889 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-11T08:34:56.988 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-11T08:34:56.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:56 smithi189 ceph-mon[32675]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:56.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:56 smithi189 ceph-mon[32675]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T08:34:56.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:56 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:56.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:56 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:56.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:56 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:34:56.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:56 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:56.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:56 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:56.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:56 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:56.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:56 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:57.002 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-09-11T08:34:57.002 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-09-11T08:34:57.002 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117466 s, 893 MB/s 2024-09-11T08:34:57.002 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-09-11T08:34:57.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[29446]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:57.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[29446]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T08:34:57.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:57.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:57.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:34:57.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:57.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:57.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:57.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:57.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[37208]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T08:34:57.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[37208]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T08:34:57.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:57.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:57.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:34:57.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:57.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:34:57.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:34:57.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:56 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:34:57.726 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_3 2024-09-11T08:34:57.978 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:57 smithi131 ceph-mon[37208]: Detected new or changed devices on smithi131 2024-09-11T08:34:57.979 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:57 smithi131 ceph-mon[37208]: Adjusting osd_memory_target on smithi131 to 16064M 2024-09-11T08:34:57.979 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:57 smithi131 ceph-mon[29446]: Detected new or changed devices on smithi131 2024-09-11T08:34:57.979 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:57 smithi131 ceph-mon[29446]: Adjusting osd_memory_target on smithi131 to 16064M 2024-09-11T08:34:57.985 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:34:58.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:57 smithi189 ceph-mon[32675]: Detected new or changed devices on smithi131 2024-09-11T08:34:58.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:57 smithi189 ceph-mon[32675]: Adjusting osd_memory_target on smithi131 to 16064M 2024-09-11T08:34:59.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:58 smithi131 ceph-mon[29446]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:34:59.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:58 smithi131 ceph-mon[37208]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:34:59.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:58 smithi189 ceph-mon[32675]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:00.097 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:59 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:35:00.097 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:59 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:35:00.097 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:34:59 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:00.097 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:59 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:35:00.097 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:59 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:35:00.097 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:34:59 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:00.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:59 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:35:00.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:59 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:35:00.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:34:59 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:01.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:00 smithi131 ceph-mon[37208]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:01.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:00 smithi131 ceph-mon[37208]: from='client.14241 v1:172.21.15.131:0/900084083' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:01.054 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:00 smithi131 ceph-mon[29446]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:01.054 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:00 smithi131 ceph-mon[29446]: from='client.14241 v1:172.21.15.131:0/900084083' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:01.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:00 smithi189 ceph-mon[32675]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:01.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:00 smithi189 ceph-mon[32675]: from='client.14241 v1:172.21.15.131:0/900084083' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:03.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:02 smithi131 ceph-mon[29446]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:03.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:02 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3646202219' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6747dbcc-3002-4d59-8546-f28ad65bc25a"}]: dispatch 2024-09-11T08:35:03.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:02 smithi131 ceph-mon[29446]: from='client.24146 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6747dbcc-3002-4d59-8546-f28ad65bc25a"}]: dispatch 2024-09-11T08:35:03.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:02 smithi131 ceph-mon[29446]: from='client.24146 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6747dbcc-3002-4d59-8546-f28ad65bc25a"}]': finished 2024-09-11T08:35:03.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:02 smithi131 ceph-mon[29446]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T08:35:03.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:02 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:03.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:02 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2228067410' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:35:03.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:02 smithi131 ceph-mon[37208]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:03.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:02 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3646202219' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6747dbcc-3002-4d59-8546-f28ad65bc25a"}]: dispatch 2024-09-11T08:35:03.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:02 smithi131 ceph-mon[37208]: from='client.24146 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6747dbcc-3002-4d59-8546-f28ad65bc25a"}]: dispatch 2024-09-11T08:35:03.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:02 smithi131 ceph-mon[37208]: from='client.24146 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6747dbcc-3002-4d59-8546-f28ad65bc25a"}]': finished 2024-09-11T08:35:03.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:02 smithi131 ceph-mon[37208]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T08:35:03.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:02 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:03.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:02 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2228067410' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:35:03.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:02 smithi189 ceph-mon[32675]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:03.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:02 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3646202219' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6747dbcc-3002-4d59-8546-f28ad65bc25a"}]: dispatch 2024-09-11T08:35:03.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:02 smithi189 ceph-mon[32675]: from='client.24146 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6747dbcc-3002-4d59-8546-f28ad65bc25a"}]: dispatch 2024-09-11T08:35:03.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:02 smithi189 ceph-mon[32675]: from='client.24146 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6747dbcc-3002-4d59-8546-f28ad65bc25a"}]': finished 2024-09-11T08:35:03.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:02 smithi189 ceph-mon[32675]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T08:35:03.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:02 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:03.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:02 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2228067410' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:35:05.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:04 smithi131 ceph-mon[37208]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:05.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:04 smithi131 ceph-mon[29446]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:05.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:04 smithi189 ceph-mon[32675]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:07.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:06 smithi131 ceph-mon[29446]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:07.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:06 smithi131 ceph-mon[37208]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:07.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:06 smithi189 ceph-mon[32675]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:09.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:08 smithi131 ceph-mon[37208]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:09.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T08:35:09.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:09.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:08 smithi131 ceph-mon[29446]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:09.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T08:35:09.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:09.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:08 smithi189 ceph-mon[32675]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:09.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T08:35:09.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:10.102 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:09 smithi131 ceph-mon[29446]: Deploying daemon osd.1 on smithi131 2024-09-11T08:35:10.102 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:09 smithi131 ceph-mon[37208]: Deploying daemon osd.1 on smithi131 2024-09-11T08:35:10.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:09 smithi189 ceph-mon[32675]: Deploying daemon osd.1 on smithi131 2024-09-11T08:35:11.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:10 smithi131 ceph-mon[29446]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:11.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:10 smithi131 ceph-mon[37208]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:11.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:10 smithi189 ceph-mon[32675]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:13.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:12 smithi131 ceph-mon[37208]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:13.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:12 smithi131 ceph-mon[29446]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:13.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:12 smithi189 ceph-mon[32675]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:14.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:13 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:14.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:13 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:14.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:13 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:14.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:13 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:14.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:13 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:14.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:13 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:14.141 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:13 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:14.142 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:13 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:14.142 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:13 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:15.357 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:15 smithi131 ceph-mon[29446]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:15.357 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:15 smithi131 ceph-mon[37208]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:15.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:15 smithi189 ceph-mon[32675]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:16.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:16 smithi131 ceph-mon[37208]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:16.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:16 smithi131 ceph-mon[29446]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:16.475 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 1 on host 'smithi131' 2024-09-11T08:35:16.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:16 smithi189 ceph-mon[32675]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:17.076 DEBUG:teuthology.orchestra.run.smithi131:osd.1> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T08:35:17.079 INFO:tasks.cephadm:Deploying osd.2 on smithi131 with /dev/vg_nvme/lv_2... 2024-09-11T08:35:17.079 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-11T08:35:17.373 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:35:17.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:17 smithi131 ceph-mon[37208]: from='osd.1 v1:172.21.15.131:6805/2577658258' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:35:17.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:17 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:17.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:17 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:17.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:17 smithi131 ceph-mon[29446]: from='osd.1 v1:172.21.15.131:6805/2577658258' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:35:17.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:17 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:17.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:17 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:17.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:17 smithi189 ceph-mon[32675]: from='osd.1 v1:172.21.15.131:6805/2577658258' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T08:35:17.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:17 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:17.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:17 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:18.484 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:18 smithi131 ceph-mon[37208]: from='osd.1 v1:172.21.15.131:6805/2577658258' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T08:35:18.484 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:18 smithi131 ceph-mon[37208]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T08:35:18.484 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:18 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:18.484 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:18 smithi131 ceph-mon[37208]: from='osd.1 v1:172.21.15.131:6805/2577658258' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:35:18.484 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:18 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:18.485 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:18 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:18.485 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:18 smithi131 ceph-mon[37208]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:18.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:18 smithi131 ceph-mon[29446]: from='osd.1 v1:172.21.15.131:6805/2577658258' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T08:35:18.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:18 smithi131 ceph-mon[29446]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T08:35:18.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:18 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:18.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:18 smithi131 ceph-mon[29446]: from='osd.1 v1:172.21.15.131:6805/2577658258' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:35:18.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:18 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:18.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:18 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:18.485 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:18 smithi131 ceph-mon[29446]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:18.486 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:35:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-1[46640]: 2024-09-11T08:35:18.196+0000 7f34ec2af640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T08:35:18.486 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:35:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-1[46640]: 2024-09-11T08:35:18.200+0000 7f34e38bd640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:35:18.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:18 smithi189 ceph-mon[32675]: from='osd.1 v1:172.21.15.131:6805/2577658258' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T08:35:18.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:18 smithi189 ceph-mon[32675]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T08:35:18.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:18 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:18.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:18 smithi189 ceph-mon[32675]: from='osd.1 v1:172.21.15.131:6805/2577658258' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:35:18.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:18 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:18.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:18 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:18.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:18 smithi189 ceph-mon[32675]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:19.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:19 smithi131 ceph-mon[29446]: from='osd.1 v1:172.21.15.131:6805/2577658258' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-11T08:35:19.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:19 smithi131 ceph-mon[29446]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T08:35:19.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:19 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:19.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:19 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:19.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:19 smithi131 ceph-mon[37208]: from='osd.1 v1:172.21.15.131:6805/2577658258' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-11T08:35:19.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:19 smithi131 ceph-mon[37208]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T08:35:19.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:19 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:19.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:19 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:19.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:19 smithi189 ceph-mon[32675]: from='osd.1 v1:172.21.15.131:6805/2577658258' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-11T08:35:19.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:19 smithi189 ceph-mon[32675]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T08:35:19.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:19 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:19.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:19 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:20.036 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-11T08:35:20.036 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-11T08:35:20.036 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-09-11T08:35:20.036 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-09-11T08:35:20.036 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124906 s, 839 MB/s 2024-09-11T08:35:20.036 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-09-11T08:35:20.396 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:20 smithi131 ceph-mon[29446]: purged_snaps scrub starts 2024-09-11T08:35:20.396 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:20 smithi131 ceph-mon[29446]: purged_snaps scrub ok 2024-09-11T08:35:20.396 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:20 smithi131 ceph-mon[29446]: osd.1 v1:172.21.15.131:6805/2577658258 boot 2024-09-11T08:35:20.396 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:20 smithi131 ceph-mon[29446]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T08:35:20.396 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:20 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:20.396 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:20 smithi131 ceph-mon[29446]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:20.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:20 smithi131 ceph-mon[37208]: purged_snaps scrub starts 2024-09-11T08:35:20.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:20 smithi131 ceph-mon[37208]: purged_snaps scrub ok 2024-09-11T08:35:20.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:20 smithi131 ceph-mon[37208]: osd.1 v1:172.21.15.131:6805/2577658258 boot 2024-09-11T08:35:20.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:20 smithi131 ceph-mon[37208]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T08:35:20.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:20 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:20.397 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:20 smithi131 ceph-mon[37208]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:20.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:20 smithi189 ceph-mon[32675]: purged_snaps scrub starts 2024-09-11T08:35:20.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:20 smithi189 ceph-mon[32675]: purged_snaps scrub ok 2024-09-11T08:35:20.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:20 smithi189 ceph-mon[32675]: osd.1 v1:172.21.15.131:6805/2577658258 boot 2024-09-11T08:35:20.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:20 smithi189 ceph-mon[32675]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T08:35:20.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:20 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:35:20.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:20 smithi189 ceph-mon[32675]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T08:35:20.597 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_2 2024-09-11T08:35:21.109 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:35:21.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:21 smithi189 ceph-mon[32675]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T08:35:21.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:21 smithi131 ceph-mon[29446]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T08:35:21.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:21 smithi131 ceph-mon[37208]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T08:35:22.434 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:22 smithi131 ceph-mon[29446]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:22.434 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:22 smithi131 ceph-mon[37208]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:22.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:22 smithi189 ceph-mon[32675]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:24.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: Detected new or changed devices on smithi131 2024-09-11T08:35:24.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:24.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:24.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:24.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:24.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: Adjusting osd_memory_target on smithi131 to 8032M 2024-09-11T08:35:24.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:24.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:24.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:24.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:24.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:24.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:24.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:24.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:24.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:35:24.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:35:24.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:24.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: Detected new or changed devices on smithi131 2024-09-11T08:35:24.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:24.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:24.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:24.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:24.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: Adjusting osd_memory_target on smithi131 to 8032M 2024-09-11T08:35:24.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:24.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:24.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:24.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:24.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:24.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:24.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:24.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:24.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:35:24.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:35:24.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:23 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:24.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: Detected new or changed devices on smithi131 2024-09-11T08:35:24.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:24.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:24.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:24.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:24.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: Adjusting osd_memory_target on smithi131 to 8032M 2024-09-11T08:35:24.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:24.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:24.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:24.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:24.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:24.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:24.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:24.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:24.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:35:24.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:35:24.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:23 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:25.232 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:24 smithi131 ceph-mon[29446]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:25.232 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:24 smithi131 ceph-mon[29446]: from='client.14268 v1:172.21.15.131:0/548806948' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:25.233 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:24 smithi131 ceph-mon[37208]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:25.233 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:24 smithi131 ceph-mon[37208]: from='client.14268 v1:172.21.15.131:0/548806948' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:25.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:24 smithi189 ceph-mon[32675]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:25.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:24 smithi189 ceph-mon[32675]: from='client.14268 v1:172.21.15.131:0/548806948' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:26.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:25 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3469871495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e63d564e-1ffc-4575-a882-89a1342664ab"}]: dispatch 2024-09-11T08:35:26.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:25 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3469871495' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e63d564e-1ffc-4575-a882-89a1342664ab"}]': finished 2024-09-11T08:35:26.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:25 smithi189 ceph-mon[32675]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T08:35:26.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:25 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:26.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:25 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3469871495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e63d564e-1ffc-4575-a882-89a1342664ab"}]: dispatch 2024-09-11T08:35:26.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:25 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3469871495' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e63d564e-1ffc-4575-a882-89a1342664ab"}]': finished 2024-09-11T08:35:26.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:25 smithi131 ceph-mon[37208]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T08:35:26.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:25 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:26.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:25 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3469871495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e63d564e-1ffc-4575-a882-89a1342664ab"}]: dispatch 2024-09-11T08:35:26.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:25 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3469871495' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e63d564e-1ffc-4575-a882-89a1342664ab"}]': finished 2024-09-11T08:35:26.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:25 smithi131 ceph-mon[29446]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T08:35:26.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:25 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:27.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:26 smithi189 ceph-mon[32675]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:27.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:26 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2677074840' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:35:27.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:26 smithi131 ceph-mon[37208]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:27.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:26 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2677074840' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:35:27.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:26 smithi131 ceph-mon[29446]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:27.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:26 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2677074840' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:35:29.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:28 smithi189 ceph-mon[32675]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:29.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:28 smithi131 ceph-mon[29446]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:29.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:28 smithi131 ceph-mon[37208]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:31.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:30 smithi131 ceph-mon[29446]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:31.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:30 smithi131 ceph-mon[37208]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:31.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:30 smithi189 ceph-mon[32675]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:33.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:32 smithi131 ceph-mon[29446]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:33.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:32 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T08:35:33.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:32 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:33.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:32 smithi131 ceph-mon[29446]: Deploying daemon osd.2 on smithi131 2024-09-11T08:35:33.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:32 smithi131 ceph-mon[37208]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:33.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:32 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T08:35:33.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:32 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:33.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:32 smithi131 ceph-mon[37208]: Deploying daemon osd.2 on smithi131 2024-09-11T08:35:33.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:32 smithi189 ceph-mon[32675]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:33.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:32 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T08:35:33.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:32 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:33.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:32 smithi189 ceph-mon[32675]: Deploying daemon osd.2 on smithi131 2024-09-11T08:35:35.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:34 smithi189 ceph-mon[32675]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:35.304 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:34 smithi131 ceph-mon[29446]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:35.305 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:34 smithi131 ceph-mon[37208]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:36.987 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:36 smithi131 ceph-mon[29446]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:37.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:36 smithi189 ceph-mon[32675]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:37.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:36 smithi131 ceph-mon[37208]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:38.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:37 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:38.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:37 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:38.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:37 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:37 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:37 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:37 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:38.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:37 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:38.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:37 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:38.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:37 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:39.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:38 smithi131 ceph-mon[29446]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:39.230 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:38 smithi131 ceph-mon[37208]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:39.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:38 smithi189 ceph-mon[32675]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:40.411 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 2 on host 'smithi131' 2024-09-11T08:35:41.057 DEBUG:teuthology.orchestra.run.smithi131:osd.2> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T08:35:41.100 INFO:tasks.cephadm:Deploying osd.3 on smithi131 with /dev/vg_nvme/lv_1... 2024-09-11T08:35:41.100 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-11T08:35:41.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[37208]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:41.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:41.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:41.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[37208]: from='osd.2 v1:172.21.15.131:6809/4000463570' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:35:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[37208]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:35:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:41.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[29446]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:41.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:41.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:41.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:41.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:41.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:41.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[29446]: from='osd.2 v1:172.21.15.131:6809/4000463570' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:35:41.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[29446]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:35:41.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:41.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:41.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:41 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:41.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:41 smithi189 ceph-mon[32675]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:41.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:41 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:41.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:41 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:41.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:41 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:41.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:41 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:41.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:41 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:41.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:41 smithi189 ceph-mon[32675]: from='osd.2 v1:172.21.15.131:6809/4000463570' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:35:41.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:41 smithi189 ceph-mon[32675]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T08:35:41.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:41 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:35:41.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:41 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:41.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:41 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:42.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:42 smithi131 ceph-mon[37208]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T08:35:42.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:42 smithi131 ceph-mon[37208]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T08:35:42.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:42 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:42.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:42 smithi131 ceph-mon[37208]: from='osd.2 v1:172.21.15.131:6809/4000463570' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:35:42.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:42 smithi131 ceph-mon[37208]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:35:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:42 smithi131 ceph-mon[29446]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T08:35:42.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:42 smithi131 ceph-mon[29446]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T08:35:42.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:42 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:42.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:42 smithi131 ceph-mon[29446]: from='osd.2 v1:172.21.15.131:6809/4000463570' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:35:42.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:42 smithi131 ceph-mon[29446]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:35:42.142 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:35:42 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-2[52627]: 2024-09-11T08:35:42.050+0000 7f7dc9194640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T08:35:42.142 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:35:42 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-2[52627]: 2024-09-11T08:35:42.053+0000 7f7dc27a6640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:35:42.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:42 smithi189 ceph-mon[32675]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T08:35:42.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:42 smithi189 ceph-mon[32675]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T08:35:42.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:42 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:42.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:42 smithi189 ceph-mon[32675]: from='osd.2 v1:172.21.15.131:6809/4000463570' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:35:42.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:42 smithi189 ceph-mon[32675]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:35:42.654 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:35:43.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:43 smithi131 ceph-mon[37208]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:43.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:43 smithi131 ceph-mon[37208]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-11T08:35:43.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:43 smithi131 ceph-mon[37208]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T08:35:43.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:43 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:43.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:43 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:43.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:43 smithi131 ceph-mon[29446]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:43.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:43 smithi131 ceph-mon[29446]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-11T08:35:43.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:43 smithi131 ceph-mon[29446]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T08:35:43.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:43 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:43.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:43 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:43.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:43 smithi189 ceph-mon[32675]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:43.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:43 smithi189 ceph-mon[32675]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-11T08:35:43.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:43 smithi189 ceph-mon[32675]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T08:35:43.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:43 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:43.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:43 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:44.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[29446]: purged_snaps scrub starts 2024-09-11T08:35:44.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[29446]: purged_snaps scrub ok 2024-09-11T08:35:44.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:44.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[29446]: osd.2 v1:172.21.15.131:6809/4000463570 boot 2024-09-11T08:35:44.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[29446]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T08:35:44.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:44.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T08:35:44.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:44.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:44.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:44.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:44.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:44.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:44.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:44.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:44.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:44.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[37208]: purged_snaps scrub starts 2024-09-11T08:35:44.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[37208]: purged_snaps scrub ok 2024-09-11T08:35:44.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:44.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[37208]: osd.2 v1:172.21.15.131:6809/4000463570 boot 2024-09-11T08:35:44.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[37208]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T08:35:44.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:44.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T08:35:44.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:44.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:44.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:44.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:44.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:44.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:44.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:44.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:44.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:44 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:44.394 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:44 smithi189 ceph-mon[32675]: purged_snaps scrub starts 2024-09-11T08:35:44.394 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:44 smithi189 ceph-mon[32675]: purged_snaps scrub ok 2024-09-11T08:35:44.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:44 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:44.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:44 smithi189 ceph-mon[32675]: osd.2 v1:172.21.15.131:6809/4000463570 boot 2024-09-11T08:35:44.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:44 smithi189 ceph-mon[32675]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T08:35:44.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:44 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:35:44.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:44 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T08:35:44.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:44 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:44.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:44 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:44.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:44 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:44.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:44 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:44.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:44 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:35:44.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:44 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:44.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:44 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:44.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:44 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:35:44.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:44 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:45.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:45 smithi131 ceph-mon[29446]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:45.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:45 smithi131 ceph-mon[29446]: Detected new or changed devices on smithi131 2024-09-11T08:35:45.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:45 smithi131 ceph-mon[29446]: Adjusting osd_memory_target on smithi131 to 5354M 2024-09-11T08:35:45.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:45 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T08:35:45.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:45 smithi131 ceph-mon[29446]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T08:35:45.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:45 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:45.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:45 smithi131 ceph-mon[37208]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:45.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:45 smithi131 ceph-mon[37208]: Detected new or changed devices on smithi131 2024-09-11T08:35:45.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:45 smithi131 ceph-mon[37208]: Adjusting osd_memory_target on smithi131 to 5354M 2024-09-11T08:35:45.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:45 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T08:35:45.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:45 smithi131 ceph-mon[37208]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T08:35:45.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:45 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-11T08:35:45.309 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-11T08:35:45.322 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-09-11T08:35:45.322 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-09-11T08:35:45.322 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011476 s, 914 MB/s 2024-09-11T08:35:45.322 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-09-11T08:35:45.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:45 smithi189 ceph-mon[32675]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T08:35:45.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:45 smithi189 ceph-mon[32675]: Detected new or changed devices on smithi131 2024-09-11T08:35:45.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:45 smithi189 ceph-mon[32675]: Adjusting osd_memory_target on smithi131 to 5354M 2024-09-11T08:35:45.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:45 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T08:35:45.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:45 smithi189 ceph-mon[32675]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T08:35:45.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:45 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:35:46.037 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_1 2024-09-11T08:35:46.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:46 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:46.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:46 smithi131 ceph-mon[37208]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T08:35:46.275 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:46 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:46.275 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:46 smithi131 ceph-mon[29446]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T08:35:46.275 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:35:46 smithi131 sudo[56722]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:35:46.275 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:35:46 smithi131 sudo[56722]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:35:46.275 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:35:46 smithi131 sudo[56722]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:35:46.275 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:35:46 smithi131 sudo[56722]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:35:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:35:46.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:46 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:35:46.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:46 smithi189 ceph-mon[32675]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T08:35:46.802 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:35:46 smithi131 sudo[56755]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:35:46.802 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:35:46 smithi131 sudo[56755]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:35:46.802 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:35:46 smithi131 sudo[56755]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:35:46.802 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:35:46 smithi131 sudo[56755]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:35:47.060 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:35:46 smithi131 sudo[56776]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:35:47.060 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:35:46 smithi131 sudo[56776]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:35:47.060 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:35:46 smithi131 sudo[56776]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:35:47.061 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:35:46 smithi131 sudo[56776]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:35:47.323 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:47 smithi131 ceph-mon[29446]: pgmap v75: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:47.323 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:47 smithi131 ceph-mon[29446]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T08:35:47.323 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:47 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:47.324 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:47 smithi131 ceph-mon[37208]: pgmap v75: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:47.324 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:47 smithi131 ceph-mon[37208]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T08:35:47.324 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:47 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:47.324 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:35:47 smithi131 sudo[56780]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:35:47.324 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:35:47 smithi131 sudo[56780]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:35:47.324 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:35:47 smithi131 sudo[56780]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:35:47.324 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:35:47 smithi131 sudo[56780]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:35:47.325 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:35:47 smithi131 sudo[56787]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:35:47.325 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:35:47 smithi131 sudo[56787]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:35:47.325 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:35:47 smithi131 sudo[56787]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:35:47.325 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:35:47 smithi131 sudo[56787]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:35:47.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:47 smithi189 ceph-mon[32675]: pgmap v75: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:47.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:47 smithi189 ceph-mon[32675]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T08:35:47.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:47 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:47.640 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:35:47 smithi131 sudo[56810]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:35:47.640 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:35:47 smithi131 sudo[56810]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:35:47.641 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:35:47 smithi131 sudo[56810]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:35:47.641 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:35:47 smithi131 sudo[56810]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:35:48.090 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:47 smithi131 sudo[56818]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:35:48.090 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:47 smithi131 sudo[56818]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:35:48.090 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:47 smithi131 sudo[56818]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:35:48.090 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:47 smithi131 sudo[56818]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:35:48.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:35:48.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:48.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:48 smithi131 sudo[56842]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:35:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:48 smithi131 sudo[56842]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:35:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:48 smithi131 sudo[56842]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:35:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:48 smithi131 sudo[56842]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:35:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:35:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:48.591 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:48 smithi189 sudo[35146]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:35:48.591 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:48 smithi189 sudo[35146]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:35:48.591 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:48 smithi189 sudo[35146]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:35:48.591 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:48 smithi189 sudo[35146]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:35:48.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:48 smithi189 sudo[35149]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:35:48.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:48 smithi189 sudo[35149]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:35:48.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:48 smithi189 sudo[35149]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:35:48.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:48 smithi189 sudo[35149]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:35:48.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:35:48.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:48.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:49.118 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:48 smithi131 sudo[56982]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T08:35:49.118 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:48 smithi131 sudo[56982]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:35:49.118 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:48 smithi131 sudo[56982]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:35:49.118 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:48 smithi131 sudo[56982]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:35:49.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: pgmap v77: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:49.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:35:49.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: from='client.24200 v1:172.21.15.131:0/1254244103' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:49.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:35:49.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:35:49.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:35:49.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:35:49.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:49.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:49.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:35:49.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:49.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T08:35:49.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:35:49.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:49.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:35:49.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:49.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:49.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 sudo[56986]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T08:35:49.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 sudo[56986]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T08:35:49.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 sudo[56986]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T08:35:49.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 sudo[56986]: pam_unix(sudo:session): session closed for user root 2024-09-11T08:35:49.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: pgmap v77: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:49.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:35:49.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: from='client.24200 v1:172.21.15.131:0/1254244103' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:49.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:35:49.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:35:49.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:35:49.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:35:49.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:49.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:49.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:35:49.393 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:49.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T08:35:49.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:35:49.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:49.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:35:49.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:49.394 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:49.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: pgmap v77: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:49.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:35:49.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: from='client.24200 v1:172.21.15.131:0/1254244103' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:35:49.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:35:49.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:35:49.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:35:49.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:35:49.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:49.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:49.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:35:49.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:49.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T08:35:49.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:35:49.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:35:49.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:35:49.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:35:49.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:35:50.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:50 smithi131 ceph-mon[29446]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:35:50.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:50 smithi131 ceph-mon[29446]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:35:50.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:50 smithi131 ceph-mon[37208]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:35:50.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:50 smithi131 ceph-mon[37208]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:35:50.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:50 smithi189 ceph-mon[32675]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T08:35:50.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:50 smithi189 ceph-mon[32675]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T08:35:51.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:51 smithi131 ceph-mon[29446]: pgmap v78: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:51 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3046784776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a1a11f-ca8a-40ea-a94b-9c6d71ab7565"}]: dispatch 2024-09-11T08:35:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:51 smithi131 ceph-mon[29446]: from='client.24209 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a1a11f-ca8a-40ea-a94b-9c6d71ab7565"}]: dispatch 2024-09-11T08:35:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:51 smithi131 ceph-mon[29446]: from='client.24209 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5a1a11f-ca8a-40ea-a94b-9c6d71ab7565"}]': finished 2024-09-11T08:35:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:51 smithi131 ceph-mon[29446]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T08:35:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:51 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:35:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:51 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1891256787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:35:51.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:51 smithi131 ceph-mon[37208]: pgmap v78: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:51.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:51 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3046784776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a1a11f-ca8a-40ea-a94b-9c6d71ab7565"}]: dispatch 2024-09-11T08:35:51.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:51 smithi131 ceph-mon[37208]: from='client.24209 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a1a11f-ca8a-40ea-a94b-9c6d71ab7565"}]: dispatch 2024-09-11T08:35:51.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:51 smithi131 ceph-mon[37208]: from='client.24209 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5a1a11f-ca8a-40ea-a94b-9c6d71ab7565"}]': finished 2024-09-11T08:35:51.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:51 smithi131 ceph-mon[37208]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T08:35:51.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:51 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:35:51.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:51 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1891256787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:35:51.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:51 smithi189 ceph-mon[32675]: pgmap v78: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:51.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:51 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3046784776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a1a11f-ca8a-40ea-a94b-9c6d71ab7565"}]: dispatch 2024-09-11T08:35:51.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:51 smithi189 ceph-mon[32675]: from='client.24209 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a1a11f-ca8a-40ea-a94b-9c6d71ab7565"}]: dispatch 2024-09-11T08:35:51.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:51 smithi189 ceph-mon[32675]: from='client.24209 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5a1a11f-ca8a-40ea-a94b-9c6d71ab7565"}]': finished 2024-09-11T08:35:51.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:51 smithi189 ceph-mon[32675]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T08:35:51.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:51 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:35:51.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:51 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1891256787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:35:53.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:53 smithi131 ceph-mon[37208]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:53 smithi131 ceph-mon[29446]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:53.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:53 smithi189 ceph-mon[32675]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:54.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:54 smithi131 ceph-mon[29446]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:54.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:54 smithi131 ceph-mon[37208]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:54.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:54 smithi189 ceph-mon[32675]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:56.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:56 smithi131 ceph-mon[29446]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:56.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T08:35:56.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:56.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:56 smithi131 ceph-mon[37208]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:56.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:56 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T08:35:56.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:56 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:56.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:56 smithi189 ceph-mon[32675]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:56.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:56 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T08:35:56.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:56 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:35:57.782 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:57 smithi131 ceph-mon[29446]: Deploying daemon osd.3 on smithi131 2024-09-11T08:35:57.782 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:57 smithi131 ceph-mon[37208]: Deploying daemon osd.3 on smithi131 2024-09-11T08:35:57.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:57 smithi189 ceph-mon[32675]: Deploying daemon osd.3 on smithi131 2024-09-11T08:35:58.621 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:35:58 smithi131 ceph-mon[29446]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:58.621 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:35:58 smithi131 ceph-mon[37208]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:35:58.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:35:58 smithi189 ceph-mon[32675]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:00.872 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:00 smithi131 ceph-mon[37208]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:00.872 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:00 smithi131 ceph-mon[29446]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:00.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:00 smithi189 ceph-mon[32675]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:01.881 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:01 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:01.881 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:01 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:01.881 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:01 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:01.882 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:01 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:01.882 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:01 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:01.882 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:01 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:01.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:01 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:01.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:01 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:01.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:01 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:02.842 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:02 smithi131 ceph-mon[29446]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:02.843 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:02 smithi131 ceph-mon[37208]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:02.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:02 smithi189 ceph-mon[32675]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:04.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:04 smithi131 ceph-mon[29446]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:04.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:04 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:04.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:04 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:04.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:04 smithi131 ceph-mon[37208]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:04.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:04 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:04.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:04 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:04.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:04 smithi189 ceph-mon[32675]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:04.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:04 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:04.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:04 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:05.015 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 3 on host 'smithi131' 2024-09-11T08:36:05.624 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:05 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:05.625 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:05 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:05.625 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:05 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:05.625 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:05 smithi131 ceph-mon[37208]: from='osd.3 v1:172.21.15.131:6813/2422737852' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:36:05.625 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:05 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:05.625 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:05 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:05.625 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:05 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:05.625 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:05 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:05.625 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:05 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:05.625 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:05 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:05.626 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:05 smithi131 ceph-mon[29446]: from='osd.3 v1:172.21.15.131:6813/2422737852' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:36:05.626 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:05 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:05.626 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:05 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:05.626 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:05 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:05.648 DEBUG:teuthology.orchestra.run.smithi131:osd.3> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T08:36:05.650 INFO:tasks.cephadm:Deploying osd.4 on smithi189 with /dev/vg_nvme/lv_4... 2024-09-11T08:36:05.651 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-11T08:36:05.900 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:36:05.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:05 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:05.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:05 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:05.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:05 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:05.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:05 smithi189 ceph-mon[32675]: from='osd.3 v1:172.21.15.131:6813/2422737852' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T08:36:05.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:05 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:05.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:05 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:05.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:05 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:06.773 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:06 smithi131 ceph-mon[37208]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:06.773 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:06 smithi131 ceph-mon[37208]: from='osd.3 v1:172.21.15.131:6813/2422737852' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T08:36:06.773 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:06 smithi131 ceph-mon[37208]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T08:36:06.774 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:06 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:06.774 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:06 smithi131 ceph-mon[37208]: from='osd.3 v1:172.21.15.131:6813/2422737852' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:36:06.774 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:06 smithi131 ceph-mon[29446]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:06.774 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:06 smithi131 ceph-mon[29446]: from='osd.3 v1:172.21.15.131:6813/2422737852' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T08:36:06.774 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:06 smithi131 ceph-mon[29446]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T08:36:06.774 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:06 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:06.774 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:06 smithi131 ceph-mon[29446]: from='osd.3 v1:172.21.15.131:6813/2422737852' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:36:06.774 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:36:06 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-3[58709]: 2024-09-11T08:36:06.554+0000 7f095c3a4640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T08:36:06.774 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:36:06 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-3[58709]: 2024-09-11T08:36:06.558+0000 7f09539b2640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:36:06.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:06 smithi189 ceph-mon[32675]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:06.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:06 smithi189 ceph-mon[32675]: from='osd.3 v1:172.21.15.131:6813/2422737852' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T08:36:06.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:06 smithi189 ceph-mon[32675]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T08:36:06.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:06 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:06.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:06 smithi189 ceph-mon[32675]: from='osd.3 v1:172.21.15.131:6813/2422737852' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-11T08:36:07.395 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-11T08:36:07.480 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-11T08:36:07.493 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-09-11T08:36:07.493 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-09-11T08:36:07.493 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114004 s, 920 MB/s 2024-09-11T08:36:07.493 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-09-11T08:36:07.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:07 smithi131 ceph-mon[29446]: from='osd.3 v1:172.21.15.131:6813/2422737852' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-11T08:36:07.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:07 smithi131 ceph-mon[29446]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T08:36:07.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:07 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:07.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:07 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:07.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:07 smithi131 ceph-mon[37208]: from='osd.3 v1:172.21.15.131:6813/2422737852' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-11T08:36:07.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:07 smithi131 ceph-mon[37208]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T08:36:07.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:07 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:07.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:07 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:07.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:07 smithi189 ceph-mon[32675]: from='osd.3 v1:172.21.15.131:6813/2422737852' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-11T08:36:07.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:07 smithi189 ceph-mon[32675]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T08:36:07.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:07 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:07.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:07 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:08.053 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_4 2024-09-11T08:36:08.302 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:36:08.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: purged_snaps scrub starts 2024-09-11T08:36:08.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: purged_snaps scrub ok 2024-09-11T08:36:08.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:08.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:08.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: osd.3 v1:172.21.15.131:6813/2422737852 boot 2024-09-11T08:36:08.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T08:36:08.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:08.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:08.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:08.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:08.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:08.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:08.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:08.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:08.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:08.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:08.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:08.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: purged_snaps scrub starts 2024-09-11T08:36:08.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: purged_snaps scrub ok 2024-09-11T08:36:08.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:08.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:08.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: osd.3 v1:172.21.15.131:6813/2422737852 boot 2024-09-11T08:36:08.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T08:36:08.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:08.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:08.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:08.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:08.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:08.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:08.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:08.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:08.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:08.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:08.643 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:08.664 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: purged_snaps scrub starts 2024-09-11T08:36:08.664 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: purged_snaps scrub ok 2024-09-11T08:36:08.664 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T08:36:08.664 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:08.664 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: osd.3 v1:172.21.15.131:6813/2422737852 boot 2024-09-11T08:36:08.665 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T08:36:08.665 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:36:08.665 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:08.665 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:08.665 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:08.665 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:08.665 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:08.665 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:08.665 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:08.665 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:08.666 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:08.666 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:09.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:09 smithi189 ceph-mon[32675]: Detected new or changed devices on smithi131 2024-09-11T08:36:09.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:09 smithi189 ceph-mon[32675]: Adjusting osd_memory_target on smithi131 to 4016M 2024-09-11T08:36:09.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:09 smithi189 ceph-mon[32675]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T08:36:09.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:09 smithi131 ceph-mon[29446]: Detected new or changed devices on smithi131 2024-09-11T08:36:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:09 smithi131 ceph-mon[29446]: Adjusting osd_memory_target on smithi131 to 4016M 2024-09-11T08:36:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:09 smithi131 ceph-mon[29446]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T08:36:09.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:09 smithi131 ceph-mon[37208]: Detected new or changed devices on smithi131 2024-09-11T08:36:09.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:09 smithi131 ceph-mon[37208]: Adjusting osd_memory_target on smithi131 to 4016M 2024-09-11T08:36:09.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:09 smithi131 ceph-mon[37208]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T08:36:10.617 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:10 smithi189 ceph-mon[32675]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:10.618 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:10 smithi189 ceph-mon[32675]: from='client.24232 v1:172.21.15.189:0/3374522964' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:10.618 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:10 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:10.618 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:10 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:10.618 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:10 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:10.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:10 smithi131 ceph-mon[29446]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:10.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:10 smithi131 ceph-mon[29446]: from='client.24232 v1:172.21.15.189:0/3374522964' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:10.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:10 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:10.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:10 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:10.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:10 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:10.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:10 smithi131 ceph-mon[37208]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:10.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:10 smithi131 ceph-mon[37208]: from='client.24232 v1:172.21.15.189:0/3374522964' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:10.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:10 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:10.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:10 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:10.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:10 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:12.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:12 smithi189 ceph-mon[32675]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:12.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:12 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.189:0/3974834492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1b6e461-b362-4c98-bdac-0e5babb0849e"}]: dispatch 2024-09-11T08:36:12.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:12 smithi189 ceph-mon[32675]: from='client.24238 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1b6e461-b362-4c98-bdac-0e5babb0849e"}]: dispatch 2024-09-11T08:36:12.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:12 smithi189 ceph-mon[32675]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1b6e461-b362-4c98-bdac-0e5babb0849e"}]': finished 2024-09-11T08:36:12.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:12 smithi189 ceph-mon[32675]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T08:36:12.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:12 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:12.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:12 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.189:0/3499325839' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:36:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:12 smithi131 ceph-mon[29446]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:12 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.189:0/3974834492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1b6e461-b362-4c98-bdac-0e5babb0849e"}]: dispatch 2024-09-11T08:36:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:12 smithi131 ceph-mon[29446]: from='client.24238 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1b6e461-b362-4c98-bdac-0e5babb0849e"}]: dispatch 2024-09-11T08:36:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:12 smithi131 ceph-mon[29446]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1b6e461-b362-4c98-bdac-0e5babb0849e"}]': finished 2024-09-11T08:36:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:12 smithi131 ceph-mon[29446]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T08:36:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:12 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:12 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.189:0/3499325839' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:36:12.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:12 smithi131 ceph-mon[37208]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:12.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:12 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.189:0/3974834492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1b6e461-b362-4c98-bdac-0e5babb0849e"}]: dispatch 2024-09-11T08:36:12.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:12 smithi131 ceph-mon[37208]: from='client.24238 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1b6e461-b362-4c98-bdac-0e5babb0849e"}]: dispatch 2024-09-11T08:36:12.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:12 smithi131 ceph-mon[37208]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1b6e461-b362-4c98-bdac-0e5babb0849e"}]': finished 2024-09-11T08:36:12.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:12 smithi131 ceph-mon[37208]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T08:36:12.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:12 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:12.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:12 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.189:0/3499325839' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:36:14.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:14 smithi131 ceph-mon[29446]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:14.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:14 smithi131 ceph-mon[37208]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:14.978 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:14 smithi189 ceph-mon[32675]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:16.888 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:16 smithi189 ceph-mon[32675]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:16.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:16 smithi131 ceph-mon[37208]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:16.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:16 smithi131 ceph-mon[29446]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:18.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:18 smithi189 ceph-mon[32675]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:18.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:18 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T08:36:18.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:18 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:18.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:18 smithi189 ceph-mon[32675]: Deploying daemon osd.4 on smithi189 2024-09-11T08:36:18.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:18 smithi131 ceph-mon[29446]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:18.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:18 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T08:36:18.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:18 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:18.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:18 smithi131 ceph-mon[29446]: Deploying daemon osd.4 on smithi189 2024-09-11T08:36:18.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:18 smithi131 ceph-mon[37208]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:18.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:18 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T08:36:18.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:18 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:18.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:18 smithi131 ceph-mon[37208]: Deploying daemon osd.4 on smithi189 2024-09-11T08:36:20.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:20 smithi189 ceph-mon[32675]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:20.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:20 smithi131 ceph-mon[29446]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:20.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:20 smithi131 ceph-mon[37208]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:22.723 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:22 smithi189 ceph-mon[32675]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:22.723 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:22 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:22.723 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:22 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:22.723 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:22 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:22.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:22 smithi131 ceph-mon[29446]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:22 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:22 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:22 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:22.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:22 smithi131 ceph-mon[37208]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:22.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:22 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:22.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:22 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:22.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:22 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:24.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:24 smithi189 ceph-mon[32675]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:24.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:24 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:24.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:24 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:24.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:24 smithi131 ceph-mon[29446]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:24.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:24 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:24.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:24 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:24.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:24 smithi131 ceph-mon[37208]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:24.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:24 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:24.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:24 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:25.242 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 4 on host 'smithi189' 2024-09-11T08:36:25.811 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:25 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:25.811 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:25 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:25.811 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:25 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:25.811 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:25 smithi189 ceph-mon[32675]: from='osd.4 v1:172.21.15.189:6800/2575698578' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:36:25.811 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:25 smithi189 ceph-mon[32675]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:36:25.811 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:25 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:25.812 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:25 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:25.812 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:25 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:25.835 DEBUG:teuthology.orchestra.run.smithi189:osd.4> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T08:36:25.838 INFO:tasks.cephadm:Deploying osd.5 on smithi189 with /dev/vg_nvme/lv_3... 2024-09-11T08:36:25.838 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-11T08:36:25.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:25 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:25.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:25 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:25.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:25 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:25.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:25 smithi131 ceph-mon[29446]: from='osd.4 v1:172.21.15.189:6800/2575698578' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:36:25.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:25 smithi131 ceph-mon[29446]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:36:25.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:25 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:25.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:25 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:25.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:25 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:25.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:25 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:25.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:25 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:25.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:25 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:25.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:25 smithi131 ceph-mon[37208]: from='osd.4 v1:172.21.15.189:6800/2575698578' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:36:25.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:25 smithi131 ceph-mon[37208]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T08:36:25.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:25 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:25.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:25 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:25.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:25 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:26.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:26 smithi189 ceph-mon[32675]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:26.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:26 smithi189 ceph-mon[32675]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T08:36:26.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:26 smithi189 ceph-mon[32675]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T08:36:26.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:26 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:26.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:26 smithi189 ceph-mon[32675]: from='osd.4 v1:172.21.15.189:6800/2575698578' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:36:26.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:26 smithi189 ceph-mon[32675]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:36:26.741 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:36:26 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4[37434]: 2024-09-11T08:36:26.630+0000 7f59e0fd9640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T08:36:26.741 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:36:26 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4[37434]: 2024-09-11T08:36:26.635+0000 7f59dbe10640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:36:26.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:26 smithi131 ceph-mon[29446]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:26.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:26 smithi131 ceph-mon[29446]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T08:36:26.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:26 smithi131 ceph-mon[29446]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T08:36:26.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:26 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:26.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:26 smithi131 ceph-mon[29446]: from='osd.4 v1:172.21.15.189:6800/2575698578' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:36:26.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:26 smithi131 ceph-mon[29446]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:36:26.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:26 smithi131 ceph-mon[37208]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:26.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:26 smithi131 ceph-mon[37208]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T08:36:26.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:26 smithi131 ceph-mon[37208]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T08:36:26.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:26 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:26.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:26 smithi131 ceph-mon[37208]: from='osd.4 v1:172.21.15.189:6800/2575698578' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:36:26.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:26 smithi131 ceph-mon[37208]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:36:27.385 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:36:27.766 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:27 smithi189 ceph-mon[32675]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-11T08:36:27.766 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:27 smithi189 ceph-mon[32675]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T08:36:27.766 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:27 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:27.766 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:27 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:27.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:27 smithi131 ceph-mon[29446]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-11T08:36:27.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:27 smithi131 ceph-mon[29446]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T08:36:27.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:27 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:27.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:27 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:27.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:27 smithi131 ceph-mon[37208]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-11T08:36:27.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:27 smithi131 ceph-mon[37208]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T08:36:27.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:27 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:27.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:27 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:28.722 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:28 smithi189 ceph-mon[32675]: purged_snaps scrub starts 2024-09-11T08:36:28.723 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:28 smithi189 ceph-mon[32675]: purged_snaps scrub ok 2024-09-11T08:36:28.723 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:28 smithi189 ceph-mon[32675]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:28.723 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:28 smithi189 ceph-mon[32675]: osd.4 v1:172.21.15.189:6800/2575698578 boot 2024-09-11T08:36:28.723 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:28 smithi189 ceph-mon[32675]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T08:36:28.723 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:28 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:28.723 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:28 smithi189 ceph-mon[32675]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T08:36:28.723 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:28 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:28.723 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:28 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:28.723 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:28 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[29446]: purged_snaps scrub starts 2024-09-11T08:36:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[29446]: purged_snaps scrub ok 2024-09-11T08:36:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[29446]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[29446]: osd.4 v1:172.21.15.189:6800/2575698578 boot 2024-09-11T08:36:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[29446]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T08:36:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[29446]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T08:36:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:28.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[37208]: purged_snaps scrub starts 2024-09-11T08:36:28.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[37208]: purged_snaps scrub ok 2024-09-11T08:36:28.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[37208]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T08:36:28.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[37208]: osd.4 v1:172.21.15.189:6800/2575698578 boot 2024-09-11T08:36:28.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[37208]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T08:36:28.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:36:28.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[37208]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T08:36:28.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:28.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:28.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:28 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:29.625 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-11T08:36:29.724 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-11T08:36:29.737 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-09-11T08:36:29.737 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-09-11T08:36:29.737 INFO:teuthology.orchestra.run.smithi189.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115868 s, 905 MB/s 2024-09-11T08:36:29.737 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 2024-09-11T08:36:29.737 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-09-11T08:36:29.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:29 smithi189 ceph-mon[32675]: Detected new or changed devices on smithi189 2024-09-11T08:36:29.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:29 smithi189 ceph-mon[32675]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-11T08:36:29.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:29 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:29.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:29 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:29.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:29 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:29.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:29 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:29.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:29 smithi189 ceph-mon[32675]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T08:36:30.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:29 smithi131 ceph-mon[29446]: Detected new or changed devices on smithi189 2024-09-11T08:36:30.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:29 smithi131 ceph-mon[29446]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-11T08:36:30.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:29 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:30.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:29 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:30.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:29 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:30.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:29 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:30.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:29 smithi131 ceph-mon[29446]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T08:36:30.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:29 smithi131 ceph-mon[37208]: Detected new or changed devices on smithi189 2024-09-11T08:36:30.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:29 smithi131 ceph-mon[37208]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-11T08:36:30.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:29 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:30.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:29 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:30.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:29 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:30.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:29 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:30.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:29 smithi131 ceph-mon[37208]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T08:36:30.289 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_3 2024-09-11T08:36:30.531 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:36:30.950 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:30 smithi189 ceph-mon[32675]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:31.061 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:30 smithi131 ceph-mon[29446]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:31.061 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:30 smithi131 ceph-mon[37208]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:32.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:32 smithi189 ceph-mon[32675]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-11T08:36:32.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:32 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:32.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:32 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:32.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:32 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:33.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:32 smithi131 ceph-mon[37208]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-11T08:36:33.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:32 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:33.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:32 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:33.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:32 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:33.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:32 smithi131 ceph-mon[29446]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-11T08:36:33.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:32 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:33.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:32 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:33.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:32 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:33.688 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:33 smithi189 ceph-mon[32675]: from='client.24259 v1:172.21.15.189:0/3266647152' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:34.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:33 smithi131 ceph-mon[29446]: from='client.24259 v1:172.21.15.189:0/3266647152' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:34.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:33 smithi131 ceph-mon[37208]: from='client.24259 v1:172.21.15.189:0/3266647152' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:34.902 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:34 smithi189 ceph-mon[32675]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T08:36:34.902 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:34 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.189:0/3991902957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "114e4e0e-dc20-454e-8456-ac3f850abed2"}]: dispatch 2024-09-11T08:36:34.902 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:34 smithi189 ceph-mon[32675]: from='client.24265 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "114e4e0e-dc20-454e-8456-ac3f850abed2"}]: dispatch 2024-09-11T08:36:34.902 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:34 smithi189 ceph-mon[32675]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "114e4e0e-dc20-454e-8456-ac3f850abed2"}]': finished 2024-09-11T08:36:34.902 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:34 smithi189 ceph-mon[32675]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T08:36:34.902 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:34 smithi131 ceph-mon[29446]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T08:36:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:34 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.189:0/3991902957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "114e4e0e-dc20-454e-8456-ac3f850abed2"}]: dispatch 2024-09-11T08:36:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:34 smithi131 ceph-mon[29446]: from='client.24265 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "114e4e0e-dc20-454e-8456-ac3f850abed2"}]: dispatch 2024-09-11T08:36:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:34 smithi131 ceph-mon[29446]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "114e4e0e-dc20-454e-8456-ac3f850abed2"}]': finished 2024-09-11T08:36:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:34 smithi131 ceph-mon[29446]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T08:36:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:35.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:34 smithi131 ceph-mon[37208]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T08:36:35.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:34 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.189:0/3991902957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "114e4e0e-dc20-454e-8456-ac3f850abed2"}]: dispatch 2024-09-11T08:36:35.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:34 smithi131 ceph-mon[37208]: from='client.24265 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "114e4e0e-dc20-454e-8456-ac3f850abed2"}]: dispatch 2024-09-11T08:36:35.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:34 smithi131 ceph-mon[37208]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "114e4e0e-dc20-454e-8456-ac3f850abed2"}]': finished 2024-09-11T08:36:35.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:34 smithi131 ceph-mon[37208]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T08:36:35.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:35.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:35 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.189:0/3979597533' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:36:36.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:35 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.189:0/3979597533' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:36:36.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:35 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.189:0/3979597533' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:36:36.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:36 smithi189 ceph-mon[32675]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-11T08:36:37.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:36 smithi131 ceph-mon[29446]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-11T08:36:37.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:36 smithi131 ceph-mon[37208]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-11T08:36:38.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:38 smithi189 ceph-mon[32675]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-11T08:36:39.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:38 smithi131 ceph-mon[29446]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-11T08:36:39.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:38 smithi131 ceph-mon[37208]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-11T08:36:40.780 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:40 smithi189 ceph-mon[32675]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:36:40.780 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:40 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T08:36:40.780 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:40 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:41.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:40 smithi131 ceph-mon[37208]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:36:41.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:40 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T08:36:41.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:40 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:41.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:40 smithi131 ceph-mon[29446]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:36:41.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:40 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T08:36:41.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:40 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:41.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:41 smithi189 ceph-mon[32675]: Deploying daemon osd.5 on smithi189 2024-09-11T08:36:41.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:41 smithi131 ceph-mon[29446]: Deploying daemon osd.5 on smithi189 2024-09-11T08:36:41.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:41 smithi131 ceph-mon[37208]: Deploying daemon osd.5 on smithi189 2024-09-11T08:36:42.881 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:42 smithi189 ceph-mon[32675]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:43.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:42 smithi131 ceph-mon[29446]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:43.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:42 smithi131 ceph-mon[37208]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:44.751 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:44 smithi189 ceph-mon[32675]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:45.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:44 smithi131 ceph-mon[29446]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:45.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:44 smithi131 ceph-mon[37208]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:45.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:45 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:45.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:45 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:45.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:45 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:46.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:45 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:45 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:45 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:46.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:45 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:46.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:45 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:46.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:45 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:47.046 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:46 smithi189 ceph-mon[32675]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:47.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:46 smithi131 ceph-mon[29446]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:47.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:46 smithi131 ceph-mon[37208]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:48.087 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 5 on host 'smithi189' 2024-09-11T08:36:48.388 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:48.388 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:48.388 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:48.388 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:48.388 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:48.388 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:48 smithi189 ceph-mon[32675]: from='osd.5 v1:172.21.15.189:6804/571798110' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:36:48.389 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:48 smithi189 ceph-mon[32675]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:36:48.389 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:48.389 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:48.389 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[29446]: from='osd.5 v1:172.21.15.189:6804/571798110' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:36:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[29446]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:36:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:48.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:48.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:48.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:48.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:48.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:48.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:48.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[37208]: from='osd.5 v1:172.21.15.189:6804/571798110' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:36:48.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[37208]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T08:36:48.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:36:48.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:48.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:48.713 DEBUG:teuthology.orchestra.run.smithi189:osd.5> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T08:36:48.716 INFO:tasks.cephadm:Deploying osd.6 on smithi189 with /dev/vg_nvme/lv_2... 2024-09-11T08:36:48.716 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-11T08:36:49.350 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:49 smithi189 ceph-mon[32675]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:49.350 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:49 smithi189 ceph-mon[32675]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T08:36:49.350 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:49 smithi189 ceph-mon[32675]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T08:36:49.350 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:49.350 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:49 smithi189 ceph-mon[32675]: from='osd.5 v1:172.21.15.189:6804/571798110' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:36:49.351 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:49 smithi189 ceph-mon[32675]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:36:49.351 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:36:49 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:36:49.122+0000 7f6d431e1640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T08:36:49.351 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:36:49 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:36:49.127+0000 7f6d3d817640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:36:49.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:49 smithi131 ceph-mon[29446]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:49.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:49 smithi131 ceph-mon[29446]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T08:36:49.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:49 smithi131 ceph-mon[29446]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T08:36:49.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:49.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:49 smithi131 ceph-mon[29446]: from='osd.5 v1:172.21.15.189:6804/571798110' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:36:49.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:49 smithi131 ceph-mon[29446]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:36:49.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:49 smithi131 ceph-mon[37208]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:49.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:49 smithi131 ceph-mon[37208]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T08:36:49.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:49 smithi131 ceph-mon[37208]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T08:36:49.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:49.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:49 smithi131 ceph-mon[37208]: from='osd.5 v1:172.21.15.189:6804/571798110' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:36:49.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:49 smithi131 ceph-mon[37208]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:36:50.357 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:50 smithi189 ceph-mon[32675]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-11T08:36:50.357 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:50 smithi189 ceph-mon[32675]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T08:36:50.357 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:50 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:50.357 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:50 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:50.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:50 smithi131 ceph-mon[29446]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-11T08:36:50.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:50 smithi131 ceph-mon[29446]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T08:36:50.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:50 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:50.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:50 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:50.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:50 smithi131 ceph-mon[37208]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-11T08:36:50.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:50 smithi131 ceph-mon[37208]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T08:36:50.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:50 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:50.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:50 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:50.541 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:36:51.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:51 smithi189 ceph-mon[32675]: purged_snaps scrub starts 2024-09-11T08:36:51.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:51 smithi189 ceph-mon[32675]: purged_snaps scrub ok 2024-09-11T08:36:51.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:51 smithi189 ceph-mon[32675]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:51.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:51 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:51.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:51 smithi189 ceph-mon[32675]: osd.5 v1:172.21.15.189:6804/571798110 boot 2024-09-11T08:36:51.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:51 smithi189 ceph-mon[32675]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T08:36:51.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:51 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:51.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:51 smithi131 ceph-mon[29446]: purged_snaps scrub starts 2024-09-11T08:36:51.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:51 smithi131 ceph-mon[29446]: purged_snaps scrub ok 2024-09-11T08:36:51.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:51 smithi131 ceph-mon[29446]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:51.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:51 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:51.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:51 smithi131 ceph-mon[29446]: osd.5 v1:172.21.15.189:6804/571798110 boot 2024-09-11T08:36:51.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:51 smithi131 ceph-mon[29446]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T08:36:51.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:51 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:51.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:51 smithi131 ceph-mon[37208]: purged_snaps scrub starts 2024-09-11T08:36:51.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:51 smithi131 ceph-mon[37208]: purged_snaps scrub ok 2024-09-11T08:36:51.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:51 smithi131 ceph-mon[37208]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T08:36:51.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:51 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:51.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:51 smithi131 ceph-mon[37208]: osd.5 v1:172.21.15.189:6804/571798110 boot 2024-09-11T08:36:51.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:51 smithi131 ceph-mon[37208]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T08:36:51.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:51 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:36:52.185 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:52 smithi189 ceph-mon[32675]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T08:36:52.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:52 smithi131 ceph-mon[37208]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T08:36:52.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:52 smithi131 ceph-mon[29446]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T08:36:52.835 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-11T08:36:52.923 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-11T08:36:52.936 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-09-11T08:36:52.937 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-09-11T08:36:52.937 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011484 s, 913 MB/s 2024-09-11T08:36:52.937 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-09-11T08:36:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[29446]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:36:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[29446]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:36:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[29446]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T08:36:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:53.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[37208]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:36:53.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[37208]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:36:53.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[37208]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T08:36:53.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:53.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:53.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:53.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:53.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:53.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:53.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:53.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:53 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:53.475 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:53 smithi189 ceph-mon[32675]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:36:53.475 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:53 smithi189 ceph-mon[32675]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:36:53.475 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:53 smithi189 ceph-mon[32675]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T08:36:53.475 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:53 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:53.476 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:53 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:53.476 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:53 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:53.476 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:53 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:36:53.476 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:53 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:53.476 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:53 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:53.476 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:53 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:36:53.476 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:53 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:36:53.500 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_2 2024-09-11T08:36:53.743 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:36:54.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:54 smithi131 ceph-mon[37208]: Detected new or changed devices on smithi189 2024-09-11T08:36:54.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:54 smithi131 ceph-mon[37208]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-11T08:36:54.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:54 smithi131 ceph-mon[29446]: Detected new or changed devices on smithi189 2024-09-11T08:36:54.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:54 smithi131 ceph-mon[29446]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-11T08:36:54.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:54 smithi189 ceph-mon[32675]: Detected new or changed devices on smithi189 2024-09-11T08:36:54.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:54 smithi189 ceph-mon[32675]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-11T08:36:55.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:55 smithi131 ceph-mon[29446]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:36:55.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:55 smithi131 ceph-mon[37208]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:36:55.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:55 smithi189 ceph-mon[32675]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:36:56.457 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:56 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:56.457 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:56 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:56.457 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:56 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:56.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:56 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:56.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:56 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:56.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:56 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:56.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:36:56.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:36:56.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:36:57.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:57 smithi189 ceph-mon[32675]: from='client.24286 v1:172.21.15.189:0/3702705913' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:57.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:57 smithi189 ceph-mon[32675]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:36:57.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:57 smithi131 ceph-mon[29446]: from='client.24286 v1:172.21.15.189:0/3702705913' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:57.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:57 smithi131 ceph-mon[29446]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:36:57.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:57 smithi131 ceph-mon[37208]: from='client.24286 v1:172.21.15.189:0/3702705913' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:36:57.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:57 smithi131 ceph-mon[37208]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:36:58.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:58 smithi189 ceph-mon[32675]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:36:58.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:58 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.189:0/3501772722' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5d731e5-cfb5-4752-a8da-6fe2ae517b5f"}]: dispatch 2024-09-11T08:36:58.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:58 smithi189 ceph-mon[32675]: from='client.24292 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5d731e5-cfb5-4752-a8da-6fe2ae517b5f"}]: dispatch 2024-09-11T08:36:58.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:58 smithi189 ceph-mon[32675]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5d731e5-cfb5-4752-a8da-6fe2ae517b5f"}]': finished 2024-09-11T08:36:58.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:58 smithi189 ceph-mon[32675]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T08:36:58.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:58 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:36:58.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:58 smithi131 ceph-mon[37208]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:36:58.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:58 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.189:0/3501772722' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5d731e5-cfb5-4752-a8da-6fe2ae517b5f"}]: dispatch 2024-09-11T08:36:58.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:58 smithi131 ceph-mon[37208]: from='client.24292 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5d731e5-cfb5-4752-a8da-6fe2ae517b5f"}]: dispatch 2024-09-11T08:36:58.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:58 smithi131 ceph-mon[37208]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5d731e5-cfb5-4752-a8da-6fe2ae517b5f"}]': finished 2024-09-11T08:36:58.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:58 smithi131 ceph-mon[37208]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T08:36:58.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:58 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:36:58.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:58 smithi131 ceph-mon[29446]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:36:58.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:58 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.189:0/3501772722' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5d731e5-cfb5-4752-a8da-6fe2ae517b5f"}]: dispatch 2024-09-11T08:36:58.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:58 smithi131 ceph-mon[29446]: from='client.24292 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5d731e5-cfb5-4752-a8da-6fe2ae517b5f"}]: dispatch 2024-09-11T08:36:58.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:58 smithi131 ceph-mon[29446]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5d731e5-cfb5-4752-a8da-6fe2ae517b5f"}]': finished 2024-09-11T08:36:58.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:58 smithi131 ceph-mon[29446]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T08:36:58.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:58 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:36:59.416 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:59 smithi131 ceph-mon[37208]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:36:59.416 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:59 smithi131 ceph-mon[37208]: Cluster is now healthy 2024-09-11T08:36:59.416 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:36:59 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.189:0/192190427' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:36:59.416 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:59 smithi131 ceph-mon[29446]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:36:59.416 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:59 smithi131 ceph-mon[29446]: Cluster is now healthy 2024-09-11T08:36:59.416 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:36:59 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.189:0/192190427' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:36:59.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:59 smithi189 ceph-mon[32675]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:36:59.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:59 smithi189 ceph-mon[32675]: Cluster is now healthy 2024-09-11T08:36:59.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:36:59 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.189:0/192190427' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:37:00.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:00 smithi189 ceph-mon[32675]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:37:00.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:00 smithi131 ceph-mon[29446]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:37:00.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:00 smithi131 ceph-mon[37208]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:37:02.841 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:02 smithi189 ceph-mon[32675]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T08:37:02.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:02 smithi131 ceph-mon[29446]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T08:37:02.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:02 smithi131 ceph-mon[37208]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T08:37:03.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:03 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T08:37:03.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:03 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:03.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:03 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T08:37:03.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:03 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:03.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:03 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T08:37:03.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:03 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:04.736 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:04 smithi189 ceph-mon[32675]: Deploying daemon osd.6 on smithi189 2024-09-11T08:37:04.737 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:04 smithi189 ceph-mon[32675]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:04.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:04 smithi131 ceph-mon[29446]: Deploying daemon osd.6 on smithi189 2024-09-11T08:37:04.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:04 smithi131 ceph-mon[29446]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:04.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:04 smithi131 ceph-mon[37208]: Deploying daemon osd.6 on smithi189 2024-09-11T08:37:04.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:04 smithi131 ceph-mon[37208]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:06.800 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:06 smithi189 ceph-mon[32675]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:06.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:06 smithi131 ceph-mon[37208]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:06.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:06 smithi131 ceph-mon[29446]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:37:08.864 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:08 smithi189 ceph-mon[32675]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:08.864 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:08.864 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:08.864 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:08 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:08.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:08 smithi131 ceph-mon[37208]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:08.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:08.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:08.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:08 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:08.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:08 smithi131 ceph-mon[29446]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:08.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:08.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:08.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:08 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:10.627 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:10 smithi189 ceph-mon[32675]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:10.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:10 smithi131 ceph-mon[29446]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:10.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:10 smithi131 ceph-mon[37208]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:11.394 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 6 on host 'smithi189' 2024-09-11T08:37:12.020 DEBUG:teuthology.orchestra.run.smithi189:osd.6> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T08:37:12.023 INFO:tasks.cephadm:Deploying osd.7 on smithi189 with /dev/vg_nvme/lv_1... 2024-09-11T08:37:12.023 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-11T08:37:12.316 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:37:12.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:12 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:12.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:12 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:12.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:12 smithi131 ceph-mon[29446]: from='osd.6 v1:172.21.15.189:6808/1725667081' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:37:12.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:12 smithi131 ceph-mon[29446]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:37:12.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:12 smithi131 ceph-mon[29446]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:12.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:12 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:12.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:12 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:12.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:12 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:12.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:12 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:12.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:12 smithi131 ceph-mon[37208]: from='osd.6 v1:172.21.15.189:6808/1725667081' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:37:12.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:12 smithi131 ceph-mon[37208]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:37:12.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:12 smithi131 ceph-mon[37208]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:12.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:12 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:12.642 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:12 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:12.674 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:12 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:12.674 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:12 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:12.674 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:12 smithi189 ceph-mon[32675]: from='osd.6 v1:172.21.15.189:6808/1725667081' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:37:12.674 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:12 smithi189 ceph-mon[32675]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T08:37:12.674 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:12 smithi189 ceph-mon[32675]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:12.674 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:12 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:12.674 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:12 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:13.545 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:13 smithi189 ceph-mon[32675]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T08:37:13.545 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:13 smithi189 ceph-mon[32675]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T08:37:13.546 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:13 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:13.546 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:13 smithi189 ceph-mon[32675]: from='osd.6 v1:172.21.15.189:6808/1725667081' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:37:13.546 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:13 smithi189 ceph-mon[32675]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:37:13.546 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:37:13 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:37:13.412+0000 7f6d26206640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T08:37:13.546 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:37:13 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:37:13.418+0000 7f6d2103d640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:37:13.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:13 smithi131 ceph-mon[29446]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T08:37:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:13 smithi131 ceph-mon[29446]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T08:37:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:13 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:13 smithi131 ceph-mon[29446]: from='osd.6 v1:172.21.15.189:6808/1725667081' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:37:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:13 smithi131 ceph-mon[29446]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:37:13.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:13 smithi131 ceph-mon[37208]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T08:37:13.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:13 smithi131 ceph-mon[37208]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T08:37:13.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:13 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:13.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:13 smithi131 ceph-mon[37208]: from='osd.6 v1:172.21.15.189:6808/1725667081' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:37:13.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:13 smithi131 ceph-mon[37208]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:37:14.596 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:14 smithi189 ceph-mon[32675]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-11T08:37:14.596 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:14 smithi189 ceph-mon[32675]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T08:37:14.596 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:14 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:14.596 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:14 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:14.596 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:14 smithi189 ceph-mon[32675]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:14.836 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-11T08:37:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:14 smithi131 ceph-mon[29446]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-11T08:37:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:14 smithi131 ceph-mon[29446]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T08:37:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:14 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:14 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:14.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:14 smithi131 ceph-mon[29446]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:14.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:14 smithi131 ceph-mon[37208]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-11T08:37:14.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:14 smithi131 ceph-mon[37208]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T08:37:14.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:14 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:14.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:14 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:14.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:14 smithi131 ceph-mon[37208]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:37:14.956 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-11T08:37:14.969 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-09-11T08:37:14.969 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-09-11T08:37:14.970 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127025 s, 825 MB/s 2024-09-11T08:37:14.970 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-09-11T08:37:15.669 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:15 smithi189 ceph-mon[32675]: purged_snaps scrub starts 2024-09-11T08:37:15.669 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:15 smithi189 ceph-mon[32675]: purged_snaps scrub ok 2024-09-11T08:37:15.669 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:15 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:15.669 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:15 smithi189 ceph-mon[32675]: osd.6 v1:172.21.15.189:6808/1725667081 boot 2024-09-11T08:37:15.669 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:15 smithi189 ceph-mon[32675]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T08:37:15.669 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:15 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:15.853 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_1 2024-09-11T08:37:15.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:15 smithi131 ceph-mon[29446]: purged_snaps scrub starts 2024-09-11T08:37:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:15 smithi131 ceph-mon[29446]: purged_snaps scrub ok 2024-09-11T08:37:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:15 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:15 smithi131 ceph-mon[29446]: osd.6 v1:172.21.15.189:6808/1725667081 boot 2024-09-11T08:37:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:15 smithi131 ceph-mon[29446]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T08:37:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:15 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:15.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:15 smithi131 ceph-mon[37208]: purged_snaps scrub starts 2024-09-11T08:37:15.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:15 smithi131 ceph-mon[37208]: purged_snaps scrub ok 2024-09-11T08:37:15.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:15 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:15.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:15 smithi131 ceph-mon[37208]: osd.6 v1:172.21.15.189:6808/1725667081 boot 2024-09-11T08:37:15.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:15 smithi131 ceph-mon[37208]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T08:37:15.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:15 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:37:16.528 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:37:16.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:16 smithi189 ceph-mon[32675]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T08:37:16.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:16 smithi189 ceph-mon[32675]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:16.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:16 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:16.649 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:16 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:16.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:16 smithi131 ceph-mon[29446]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T08:37:16.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:16 smithi131 ceph-mon[29446]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:16.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:16 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:16.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:16 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:16.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:16 smithi131 ceph-mon[37208]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T08:37:16.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:16 smithi131 ceph-mon[37208]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:16.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:16.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:16 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:17.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:17 smithi189 ceph-mon[32675]: Detected new or changed devices on smithi189 2024-09-11T08:37:17.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:17 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:17.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:17 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:17.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:17 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:17.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:17 smithi189 ceph-mon[32675]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-11T08:37:17.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:17 smithi189 ceph-mon[32675]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T08:37:17.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:17 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:17.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:17 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:17.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:17 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:17.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:17 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:17.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:17 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:17.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:17 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:17.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:17 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:17.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:17 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[29446]: Detected new or changed devices on smithi189 2024-09-11T08:37:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[29446]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-11T08:37:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[29446]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T08:37:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:17.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:17.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:17.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:17.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[37208]: Detected new or changed devices on smithi189 2024-09-11T08:37:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[37208]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-11T08:37:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[37208]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T08:37:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:17.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:17.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:17.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:17.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:17.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:17.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:17.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:17 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:18.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:18 smithi189 ceph-mon[32675]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:18.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:18 smithi189 ceph-mon[32675]: from='client.24313 v1:172.21.15.189:0/1445874451' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:37:18.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:18 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:37:18.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:18 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:37:18.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:18 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:18.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:18 smithi131 ceph-mon[37208]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:18.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:18 smithi131 ceph-mon[37208]: from='client.24313 v1:172.21.15.189:0/1445874451' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:37:18.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:18 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:37:18.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:18 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:37:18.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:18 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:18.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:18 smithi131 ceph-mon[29446]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:18.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:18 smithi131 ceph-mon[29446]: from='client.24313 v1:172.21.15.189:0/1445874451' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:37:18.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:18 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:37:18.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:18 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:37:18.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:18 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:20 smithi131 ceph-mon[29446]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:20 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.189:0/954651769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99012398-2a41-475d-918c-421d581f11a8"}]: dispatch 2024-09-11T08:37:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:20 smithi131 ceph-mon[29446]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99012398-2a41-475d-918c-421d581f11a8"}]: dispatch 2024-09-11T08:37:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:20 smithi131 ceph-mon[29446]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99012398-2a41-475d-918c-421d581f11a8"}]': finished 2024-09-11T08:37:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:20 smithi131 ceph-mon[29446]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T08:37:20.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:20 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:37:20.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:20 smithi131 ceph-mon[37208]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:20.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:20 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.189:0/954651769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99012398-2a41-475d-918c-421d581f11a8"}]: dispatch 2024-09-11T08:37:20.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:20 smithi131 ceph-mon[37208]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99012398-2a41-475d-918c-421d581f11a8"}]: dispatch 2024-09-11T08:37:20.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:20 smithi131 ceph-mon[37208]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99012398-2a41-475d-918c-421d581f11a8"}]': finished 2024-09-11T08:37:20.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:20 smithi131 ceph-mon[37208]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T08:37:20.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:20 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:37:20.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:20 smithi189 ceph-mon[32675]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:20.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:20 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.189:0/954651769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99012398-2a41-475d-918c-421d581f11a8"}]: dispatch 2024-09-11T08:37:20.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:20 smithi189 ceph-mon[32675]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99012398-2a41-475d-918c-421d581f11a8"}]: dispatch 2024-09-11T08:37:20.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:20 smithi189 ceph-mon[32675]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99012398-2a41-475d-918c-421d581f11a8"}]': finished 2024-09-11T08:37:20.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:20 smithi189 ceph-mon[32675]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T08:37:20.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:20 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:37:21.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:21 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.189:0/1290144389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:37:21.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:21 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.189:0/1290144389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:37:21.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:21 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.189:0/1290144389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:37:22.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:22 smithi131 ceph-mon[29446]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:22.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:22 smithi131 ceph-mon[37208]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:22.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:22 smithi189 ceph-mon[32675]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:24.789 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:24 smithi189 ceph-mon[32675]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:24.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:24 smithi131 ceph-mon[29446]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:24.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:24 smithi131 ceph-mon[37208]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:26.813 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:26 smithi189 ceph-mon[32675]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:26.813 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:26 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:37:26.813 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:26 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:26.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:26 smithi131 ceph-mon[29446]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:26.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:26 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:37:26.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:26 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:26.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:26 smithi131 ceph-mon[37208]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:26.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:26 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:37:26.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:26 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:27.848 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:27 smithi189 ceph-mon[32675]: Deploying daemon osd.7 on smithi189 2024-09-11T08:37:27.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:27 smithi131 ceph-mon[29446]: Deploying daemon osd.7 on smithi189 2024-09-11T08:37:27.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:27 smithi131 ceph-mon[37208]: Deploying daemon osd.7 on smithi189 2024-09-11T08:37:28.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:28 smithi131 ceph-mon[29446]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:28.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:28 smithi131 ceph-mon[37208]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:28.983 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:28 smithi189 ceph-mon[32675]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:30.802 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:30 smithi189 ceph-mon[32675]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:30.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:30 smithi131 ceph-mon[29446]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:30.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:30 smithi131 ceph-mon[37208]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:31.858 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:31 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:31.858 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:31 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:31.858 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:31 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:31.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:31 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:31.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:31 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:31.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:31 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:31.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:31 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:31.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:31 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:31.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:31 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:32.890 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:32 smithi189 ceph-mon[32675]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:32.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:32 smithi131 ceph-mon[29446]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:32.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:32 smithi131 ceph-mon[37208]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:34.467 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 7 on host 'smithi189' 2024-09-11T08:37:35.110 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:34 smithi189 ceph-mon[32675]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:35.110 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:35.110 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:35.110 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:35.111 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:35.111 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:35.111 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:34 smithi189 ceph-mon[32675]: from='osd.7 v1:172.21.15.189:6812/1579379153' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:37:35.111 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:34 smithi189 ceph-mon[32675]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:37:35.111 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:35.111 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:35.111 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:34 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:35.134 DEBUG:teuthology.orchestra.run.smithi189:osd.7> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T08:37:35.136 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-11T08:37:35.137 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T08:37:35.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[37208]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:35.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:35.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:35.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:35.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:35.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:35.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[37208]: from='osd.7 v1:172.21.15.189:6812/1579379153' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:37:35.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[37208]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:37:35.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:35.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:35.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[29446]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:35.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:35.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:35.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:35.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:35.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[29446]: from='osd.7 v1:172.21.15.189:6812/1579379153' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:37:35.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[29446]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:37:35.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:35.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:35.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:34 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:35.397 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:37:36.326 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:36 smithi189 ceph-mon[32675]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:37:36.326 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:36 smithi189 ceph-mon[32675]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T08:37:36.326 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:36 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:37:36.326 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:36 smithi189 ceph-mon[32675]: from='osd.7 v1:172.21.15.189:6812/1579379153' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:37:36.326 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:36 smithi189 ceph-mon[32675]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:37:36.326 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:36 smithi189 ceph-mon[32675]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:36.327 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:37:36 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:37:36.201+0000 7fa9f2ea5640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T08:37:36.327 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:37:36 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:37:36.207+0000 7fa9ee4dd640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:37:36.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:36 smithi131 ceph-mon[37208]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:37:36.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:36 smithi131 ceph-mon[37208]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T08:37:36.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:36 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:37:36.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:36 smithi131 ceph-mon[37208]: from='osd.7 v1:172.21.15.189:6812/1579379153' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:37:36.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:36 smithi131 ceph-mon[37208]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:37:36.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:36 smithi131 ceph-mon[37208]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:36.531 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:36 smithi131 ceph-mon[29446]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:37:36.531 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:36 smithi131 ceph-mon[29446]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T08:37:36.531 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:36 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:37:36.531 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:36 smithi131 ceph-mon[29446]: from='osd.7 v1:172.21.15.189:6812/1579379153' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:37:36.531 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:36 smithi131 ceph-mon[29446]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-11T08:37:36.531 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:36 smithi131 ceph-mon[29446]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:37:37.201 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:37:37.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:37 smithi131 ceph-mon[37208]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-11T08:37:37.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:37 smithi131 ceph-mon[37208]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T08:37:37.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:37 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:37:37.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:37 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:37:37.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:37 smithi131 ceph-mon[37208]: osd.7 v1:172.21.15.189:6812/1579379153 boot 2024-09-11T08:37:37.407 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:37 smithi131 ceph-mon[37208]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T08:37:37.407 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:37 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:37:37.407 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:37 smithi131 ceph-mon[29446]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-11T08:37:37.407 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:37 smithi131 ceph-mon[29446]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T08:37:37.407 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:37 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:37:37.407 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:37 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:37:37.407 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:37 smithi131 ceph-mon[29446]: osd.7 v1:172.21.15.189:6812/1579379153 boot 2024-09-11T08:37:37.407 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:37 smithi131 ceph-mon[29446]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T08:37:37.407 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:37 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:37:37.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:37 smithi189 ceph-mon[32675]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-11T08:37:37.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:37 smithi189 ceph-mon[32675]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T08:37:37.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:37 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:37:37.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:37 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:37:37.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:37 smithi189 ceph-mon[32675]: osd.7 v1:172.21.15.189:6812/1579379153 boot 2024-09-11T08:37:37.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:37 smithi189 ceph-mon[32675]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T08:37:37.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:37 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:37:37.827 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":48,"num_osds":8,"num_up_osds":8,"osd_up_since":1726043857,"num_in_osds":8,"osd_in_since":1726043840,"num_remapped_pgs":1} 2024-09-11T08:37:37.828 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:37:38.122 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:37:38.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: purged_snaps scrub starts 2024-09-11T08:37:38.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: purged_snaps scrub ok 2024-09-11T08:37:38.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1576090947' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:37:38.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:38.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: Detected new or changed devices on smithi189 2024-09-11T08:37:38.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:38.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:38.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:38.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:38.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:38.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:38.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-11T08:37:38.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:38.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:38.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:38.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:38.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:38 smithi189 ceph-mon[32675]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T08:37:38.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: purged_snaps scrub starts 2024-09-11T08:37:38.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: purged_snaps scrub ok 2024-09-11T08:37:38.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1576090947' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:37:38.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:38.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: Detected new or changed devices on smithi189 2024-09-11T08:37:38.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:38.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:38.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:38.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:38.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:38.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:38.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-11T08:37:38.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:38.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:38.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:38.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:38.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[37208]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T08:37:38.531 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: purged_snaps scrub starts 2024-09-11T08:37:38.531 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: purged_snaps scrub ok 2024-09-11T08:37:38.531 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1576090947' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:37:38.531 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:38.531 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: Detected new or changed devices on smithi189 2024-09-11T08:37:38.532 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:38.532 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:38.532 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:38.532 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:38.532 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:38.532 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:37:38.532 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-11T08:37:38.532 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:38.532 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:38.532 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:38.532 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:38.532 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:38 smithi131 ceph-mon[29446]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T08:37:39.800 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:37:39.800 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":50,"fsid":"50e4d0bc-7018-11ef-bcea-c7b262605968","created":"2024-09-11T08:32:52.578895+0000","modified":"2024-09-11T08:37:39.185653+0000","last_up_change":"2024-09-11T08:37:37.187181+0000","last_in_change":"2024-09-11T08:37:20.495590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:35:43.725581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"2a4b0f7e-5c59-472b-ac6b-fefc3f09e091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6801","nonce":1075394441}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6802","nonce":1075394441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6804","nonce":1075394441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6803","nonce":1075394441}]},"public_addr":"172.21.15.131:6801/1075394441","cluster_addr":"172.21.15.131:6802/1075394441","heartbeat_back_addr":"172.21.15.131:6804/1075394441","heartbeat_front_addr":"172.21.15.131:6803/1075394441","state":["exists","up"]},{"osd":1,"uuid":"6747dbcc-3002-4d59-8546-f28ad65bc25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6805","nonce":2577658258}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6806","nonce":2577658258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6808","nonce":2577658258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6807","nonce":2577658258}]},"public_addr":"172.21.15.131:6805/2577658258","cluster_addr":"172.21.15.131:6806/2577658258","heartbeat_back_addr":"172.21.15.131:6808/2577658258","heartbeat_front_addr":"172.21.15.131:6807/2577658258","state":["exists","up"]},{"osd":2,"uuid":"e63d564e-1ffc-4575-a882-89a1342664ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6809","nonce":4000463570}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6810","nonce":4000463570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6812","nonce":4000463570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6811","nonce":4000463570}]},"public_addr":"172.21.15.131:6809/4000463570","cluster_addr":"172.21.15.131:6810/4000463570","heartbeat_back_addr":"172.21.15.131:6812/4000463570","heartbeat_front_addr":"172.21.15.131:6811/4000463570","state":["exists","up"]},{"osd":3,"uuid":"b5a1a11f-ca8a-40ea-a94b-9c6d71ab7565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6813","nonce":2422737852}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6814","nonce":2422737852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6816","nonce":2422737852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6815","nonce":2422737852}]},"public_addr":"172.21.15.131:6813/2422737852","cluster_addr":"172.21.15.131:6814/2422737852","heartbeat_back_addr":"172.21.15.131:6816/2422737852","heartbeat_front_addr":"172.21.15.131:6815/2422737852","state":["exists","up"]},{"osd":4,"uuid":"f1b6e461-b362-4c98-bdac-0e5babb0849e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6800","nonce":2575698578}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6801","nonce":2575698578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6803","nonce":2575698578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6802","nonce":2575698578}]},"public_addr":"172.21.15.189:6800/2575698578","cluster_addr":"172.21.15.189:6801/2575698578","heartbeat_back_addr":"172.21.15.189:6803/2575698578","heartbeat_front_addr":"172.21.15.189:6802/2575698578","state":["exists","up"]},{"osd":5,"uuid":"114e4e0e-dc20-454e-8456-ac3f850abed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6804","nonce":571798110}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6805","nonce":571798110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6807","nonce":571798110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6806","nonce":571798110}]},"public_addr":"172.21.15.189:6804/571798110","cluster_addr":"172.21.15.189:6805/571798110","heartbeat_back_addr":"172.21.15.189:6807/571798110","heartbeat_front_addr":"172.21.15.189:6806/571798110","state":["exists","up"]},{"osd":6,"uuid":"f5d731e5-cfb5-4752-a8da-6fe2ae517b5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6808","nonce":1725667081}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6809","nonce":1725667081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6811","nonce":1725667081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6810","nonce":1725667081}]},"public_addr":"172.21.15.189:6808/1725667081","cluster_addr":"172.21.15.189:6809/1725667081","heartbeat_back_addr":"172.21.15.189:6811/1725667081","heartbeat_front_addr":"172.21.15.189:6810/1725667081","state":["exists","up"]},{"osd":7,"uuid":"99012398-2a41-475d-918c-421d581f11a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6812","nonce":1579379153}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6813","nonce":1579379153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6815","nonce":1579379153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6814","nonce":1579379153}]},"public_addr":"172.21.15.189:6812/1579379153","cluster_addr":"172.21.15.189:6813/1579379153","heartbeat_back_addr":"172.21.15.189:6815/1579379153","heartbeat_front_addr":"172.21.15.189:6814/1579379153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:34:52.816843+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:35:17.202169+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:35:41.237351+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:36:05.630435+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:36:26.230762+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:36:49.043341+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:37:12.486687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:37:35.285916+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/1949349877":"2024-09-12T08:33:35.432760+0000","172.21.15.131:0/957761929":"2024-09-12T08:33:35.432760+0000","172.21.15.131:0/2175368038":"2024-09-12T08:33:35.432760+0000","172.21.15.131:0/470181260":"2024-09-12T08:33:13.813660+0000","172.21.15.131:6800/3956075890":"2024-09-12T08:33:13.813660+0000","172.21.15.131:6800/1673105920":"2024-09-12T08:33:35.432760+0000","172.21.15.131:0/1761772131":"2024-09-12T08:33:13.813660+0000","172.21.15.131:0/2726656335":"2024-09-12T08:33:13.813660+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:37:40.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:40 smithi131 ceph-mon[29446]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T08:37:40.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:40 smithi131 ceph-mon[29446]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:40.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:40 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/247305857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:37:40.464 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:40 smithi131 ceph-mon[37208]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T08:37:40.464 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:40 smithi131 ceph-mon[37208]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:40.464 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:40 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/247305857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:37:40.465 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T08:35:43.725581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-11T08:37:40.466 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T08:37:40.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:40 smithi189 ceph-mon[32675]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T08:37:40.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:40 smithi189 ceph-mon[32675]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:40.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:40 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/247305857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:37:40.710 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:37:42.350 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-09-11T08:37:42.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:42 smithi131 ceph-mon[29446]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:42.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:42 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3074509782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:37:42.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:42 smithi131 ceph-mon[37208]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:42.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:42 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3074509782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:37:42.981 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi131 2024-09-11T08:37:42.982 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi131=foo.a' 2024-09-11T08:37:42.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:42 smithi189 ceph-mon[32675]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:42.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:42 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3074509782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:37:43.239 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:37:44.579 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:44 smithi189 ceph-mon[32675]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:44.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:44 smithi131 ceph-mon[29446]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:44.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:44 smithi131 ceph-mon[37208]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:44.983 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled rgw.foo.a update... 2024-09-11T08:37:45.588 DEBUG:teuthology.orchestra.run.smithi131:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-11T08:37:45.591 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi189 2024-09-11T08:37:45.591 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-11T08:37:45.882 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:37:46.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:45 smithi189 ceph-mon[32675]: from='client.24355 v1:172.21.15.189:0/334181000' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi131=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:37:46.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:45 smithi189 ceph-mon[32675]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-09-11T08:37:46.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:45 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:46.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:45 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:46.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:45 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:46.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:45 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:46.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:45 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:46.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:45 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:37:46.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:45 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:37:46.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:45 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:46.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:45 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:46.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:45 smithi189 ceph-mon[32675]: Deploying daemon rgw.foo.a on smithi131 2024-09-11T08:37:46.280 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[37208]: from='client.24355 v1:172.21.15.189:0/334181000' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi131=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:37:46.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[37208]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-09-11T08:37:46.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:46.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:46.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:46.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:46.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:46.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:37:46.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:37:46.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:46.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:46.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[37208]: Deploying daemon rgw.foo.a on smithi131 2024-09-11T08:37:46.282 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[29446]: from='client.24355 v1:172.21.15.189:0/334181000' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi131=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:37:46.282 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[29446]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-09-11T08:37:46.282 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:46.282 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:46.282 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:46.282 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:46.282 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:46.283 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:37:46.283 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:37:46.283 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:46.283 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:46.283 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:45 smithi131 ceph-mon[29446]: Deploying daemon rgw.foo.a on smithi131 2024-09-11T08:37:47.000 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 11 08:37:46 smithi131 podman[63784]: 2024-09-11 08:37:46.873762681 +0000 UTC m=+0.123127875 container create e5be8d02b62001f4a9274473cc7b13bbda90d7dfc1d06669274254a051b68246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:37:47.175 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:47 smithi189 ceph-mon[32675]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:47.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:47 smithi131 ceph-mon[37208]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:47.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:47 smithi131 ceph-mon[29446]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:47.393 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 11 08:37:47 smithi131 podman[63784]: 2024-09-11 08:37:47.135320299 +0000 UTC m=+0.384685460 container init e5be8d02b62001f4a9274473cc7b13bbda90d7dfc1d06669274254a051b68246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-11T08:37:47.393 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 11 08:37:47 smithi131 podman[63784]: 2024-09-11 08:37:47.138191072 +0000 UTC m=+0.387556226 container start e5be8d02b62001f4a9274473cc7b13bbda90d7dfc1d06669274254a051b68246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-rgw-foo-a, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-11T08:37:47.393 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 11 08:37:47 smithi131 bash[63784]: e5be8d02b62001f4a9274473cc7b13bbda90d7dfc1d06669274254a051b68246 2024-09-11T08:37:47.393 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 11 08:37:47 smithi131 systemd[1]: Started Ceph rgw.foo.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:37:48.008 INFO:teuthology.orchestra.run.smithi189.stderr:pool 'datapool' created 2024-09-11T08:37:48.627 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:48.627 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:48.628 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:48.628 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:48 smithi189 ceph-mon[32675]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-09-11T08:37:48.628 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:48.628 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:48.628 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:48 smithi189 ceph-mon[32675]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:48.628 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:48 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:48.628 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:48 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.189:0/1526060296' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:37:48.628 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:48 smithi189 ceph-mon[32675]: from='client.24370 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:37:48.628 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:48 smithi189 ceph-mon[32675]: from='client.24370 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:37:48.628 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:48 smithi189 ceph-mon[32675]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T08:37:48.628 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:48 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:37:48.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:48.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:48.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:48.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[37208]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-09-11T08:37:48.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:48.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:48.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[37208]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:48.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:48.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.189:0/1526060296' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:37:48.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[37208]: from='client.24370 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:37:48.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[37208]: from='client.24370 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:37:48.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[37208]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T08:37:48.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:37:48.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:48.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:48.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:48.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[29446]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-09-11T08:37:48.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:48.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:48.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[29446]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:48.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:37:48.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.189:0/1526060296' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:37:48.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[29446]: from='client.24370 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:37:48.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[29446]: from='client.24370 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:37:48.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[29446]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T08:37:48.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:48 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:37:48.657 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-11T08:37:48.904 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:37:50.115 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:50.115 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:50.115 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:50.115 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:49 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:50.115 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:49 smithi189 ceph-mon[32675]: Checking dashboard <-> RGW credentials 2024-09-11T08:37:50.115 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:49 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:37:50.115 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:49 smithi189 ceph-mon[32675]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T08:37:50.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:50.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:50.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:50.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:49 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:50.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:49 smithi131 ceph-mon[37208]: Checking dashboard <-> RGW credentials 2024-09-11T08:37:50.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:49 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:37:50.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:49 smithi131 ceph-mon[37208]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T08:37:50.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:50.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:50.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:50.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:49 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:37:50.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:49 smithi131 ceph-mon[29446]: Checking dashboard <-> RGW credentials 2024-09-11T08:37:50.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:49 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:37:50.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:49 smithi131 ceph-mon[29446]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T08:37:51.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[29446]: pgmap v169: 36 pgs: 4 creating+peering, 27 unknown, 5 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[29446]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[29446]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T08:37:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:37:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/331405991' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:37:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[29446]: from='client.24386 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:37:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.189:0/1914092918' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:37:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[29446]: from='client.24400 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:37:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:51.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[37208]: pgmap v169: 36 pgs: 4 creating+peering, 27 unknown, 5 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:51.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[37208]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:51.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[37208]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T08:37:51.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:37:51.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/331405991' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:37:51.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[37208]: from='client.24386 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:37:51.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.189:0/1914092918' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:37:51.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[37208]: from='client.24400 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:37:51.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:50 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:51.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:50 smithi189 ceph-mon[32675]: pgmap v169: 36 pgs: 4 creating+peering, 27 unknown, 5 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:37:51.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:50 smithi189 ceph-mon[32675]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:37:51.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:50 smithi189 ceph-mon[32675]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T08:37:51.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:50 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:37:51.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:50 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/331405991' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:37:51.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:50 smithi189 ceph-mon[32675]: from='client.24386 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:37:51.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:50 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.189:0/1914092918' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:37:51.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:50 smithi189 ceph-mon[32675]: from='client.24400 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:37:51.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:50 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:52.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:52 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:37:52.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:52 smithi131 ceph-mon[37208]: from='client.24386 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:37:52.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:52 smithi131 ceph-mon[37208]: from='client.24400 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:37:52.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:52 smithi131 ceph-mon[37208]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:37:52.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:52 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:37:52.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:52 smithi131 ceph-mon[29446]: from='client.24386 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:37:52.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:52 smithi131 ceph-mon[29446]: from='client.24400 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:37:52.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:52 smithi131 ceph-mon[29446]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:37:52.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:52 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:37:52.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:52 smithi189 ceph-mon[32675]: from='client.24386 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:37:52.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:52 smithi189 ceph-mon[32675]: from='client.24400 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:37:52.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:52 smithi189 ceph-mon[32675]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:37:53.340 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:53 smithi189 ceph-mon[32675]: pgmap v172: 68 pgs: 2 creating+activating, 17 creating+peering, 24 unknown, 25 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:37:53.341 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:53 smithi189 ceph-mon[32675]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:37:53.341 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:53 smithi189 ceph-mon[32675]: Cluster is now healthy 2024-09-11T08:37:53.341 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:53 smithi189 ceph-mon[32675]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:37:53.341 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:53 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:37:53.341 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:53 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/331405991' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:37:53.341 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:53 smithi189 ceph-mon[32675]: from='client.24386 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:37:53.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:53 smithi131 ceph-mon[37208]: pgmap v172: 68 pgs: 2 creating+activating, 17 creating+peering, 24 unknown, 25 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:37:53.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:53 smithi131 ceph-mon[37208]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:37:53.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:53 smithi131 ceph-mon[37208]: Cluster is now healthy 2024-09-11T08:37:53.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:53 smithi131 ceph-mon[37208]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:37:53.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:53 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:37:53.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:53 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/331405991' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:37:53.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:53 smithi131 ceph-mon[37208]: from='client.24386 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:37:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:53 smithi131 ceph-mon[29446]: pgmap v172: 68 pgs: 2 creating+activating, 17 creating+peering, 24 unknown, 25 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:37:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:53 smithi131 ceph-mon[29446]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:37:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:53 smithi131 ceph-mon[29446]: Cluster is now healthy 2024-09-11T08:37:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:53 smithi131 ceph-mon[29446]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:37:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:53 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:37:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:53 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/331405991' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:37:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:53 smithi131 ceph-mon[29446]: from='client.24386 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:37:53.621 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.189 --placement '1;smithi189=iscsi.a' 2024-09-11T08:37:53.864 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:37:54.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:54 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:37:54.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:54 smithi189 ceph-mon[32675]: from='client.24386 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:37:54.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:54 smithi189 ceph-mon[32675]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:37:54.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:54 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:37:54.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:54 smithi131 ceph-mon[29446]: from='client.24386 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:37:54.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:54 smithi131 ceph-mon[29446]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:37:54.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:54 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3920969160' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:37:54.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:54 smithi131 ceph-mon[37208]: from='client.24386 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:37:54.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:54 smithi131 ceph-mon[37208]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:37:55.311 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:55 smithi189 ceph-mon[32675]: pgmap v175: 100 pgs: 2 creating+activating, 25 creating+peering, 32 unknown, 41 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-11T08:37:55.311 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:55 smithi189 ceph-mon[32675]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:37:55.311 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:55 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3982199361' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:37:55.311 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:55 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2650549601' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:37:55.311 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:55 smithi189 ceph-mon[32675]: from='client.24412 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:37:55.311 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:55 smithi189 ceph-mon[32675]: from='client.24406 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:37:55.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:55 smithi131 ceph-mon[37208]: pgmap v175: 100 pgs: 2 creating+activating, 25 creating+peering, 32 unknown, 41 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-11T08:37:55.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:55 smithi131 ceph-mon[37208]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:37:55.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:55 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3982199361' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:37:55.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:55 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2650549601' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:37:55.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:55 smithi131 ceph-mon[37208]: from='client.24412 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:37:55.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:55 smithi131 ceph-mon[37208]: from='client.24406 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:37:55.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:55 smithi131 ceph-mon[29446]: pgmap v175: 100 pgs: 2 creating+activating, 25 creating+peering, 32 unknown, 41 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-11T08:37:55.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:55 smithi131 ceph-mon[29446]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:37:55.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:55 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3982199361' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:37:55.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:55 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2650549601' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:37:55.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:55 smithi131 ceph-mon[29446]: from='client.24412 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:37:55.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:55 smithi131 ceph-mon[29446]: from='client.24406 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:37:55.582 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled iscsi.datapool update... 2024-09-11T08:37:56.140 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:56 smithi189 ceph-mon[32675]: from='client.24412 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:37:56.140 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:56 smithi189 ceph-mon[32675]: from='client.24406 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:37:56.140 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:56 smithi189 ceph-mon[32675]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:37:56.140 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:56 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3982199361' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:37:56.140 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:56 smithi189 ceph-mon[32675]: from='client.24412 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:37:56.140 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:56 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2650549601' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:37:56.140 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:56 smithi189 ceph-mon[32675]: from='client.24406 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:37:56.140 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:56 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:56.141 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:56 smithi189 ceph-mon[32675]: from='client.24412 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:37:56.141 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:56 smithi189 ceph-mon[32675]: from='client.24406 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:37:56.141 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:56 smithi189 ceph-mon[32675]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:37:56.168 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-11T08:37:56.168 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-11T08:37:56.168 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T08:37:56.202 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-11T08:37:56.202 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T08:37:56.232 DEBUG:teuthology.orchestra.run.smithi189:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-11T08:37:56.274 INFO:tasks.cephadm:Adding prometheus.a on smithi189 2024-09-11T08:37:56.274 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi189=a' 2024-09-11T08:37:56.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[37208]: from='client.24412 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:37:56.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[37208]: from='client.24406 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:37:56.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[37208]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:37:56.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3982199361' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:37:56.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[37208]: from='client.24412 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:37:56.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2650549601' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:37:56.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[37208]: from='client.24406 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:37:56.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:56.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[37208]: from='client.24412 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:37:56.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[37208]: from='client.24406 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:37:56.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[37208]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:37:56.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[29446]: from='client.24412 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:37:56.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[29446]: from='client.24406 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:37:56.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[29446]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:37:56.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3982199361' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:37:56.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[29446]: from='client.24412 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:37:56.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2650549601' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:37:56.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[29446]: from='client.24406 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:37:56.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:56.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[29446]: from='client.24412 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:37:56.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[29446]: from='client.24406 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:37:56.393 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:56 smithi131 ceph-mon[29446]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:37:56.578 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:37:57.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:57 smithi189 ceph-mon[32675]: pgmap v178: 132 pgs: 3 creating+activating, 15 creating+peering, 35 unknown, 79 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-11T08:37:57.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:57 smithi189 ceph-mon[32675]: from='client.24418 v1:172.21.15.189:0/1930629186' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.189", "placement": "1;smithi189=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:37:57.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:57 smithi189 ceph-mon[32675]: Saving service iscsi.datapool spec with placement smithi189=iscsi.a;count:1 2024-09-11T08:37:57.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:57 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:57.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:57 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:57.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:57 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:57.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:57 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:37:57.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:57 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:37:57.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:57 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:57.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[37208]: pgmap v178: 132 pgs: 3 creating+activating, 15 creating+peering, 35 unknown, 79 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-11T08:37:57.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[37208]: from='client.24418 v1:172.21.15.189:0/1930629186' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.189", "placement": "1;smithi189=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:37:57.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[37208]: Saving service iscsi.datapool spec with placement smithi189=iscsi.a;count:1 2024-09-11T08:37:57.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:57.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:57.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:57.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:37:57.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:37:57.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:57.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[29446]: pgmap v178: 132 pgs: 3 creating+activating, 15 creating+peering, 35 unknown, 79 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-11T08:37:57.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[29446]: from='client.24418 v1:172.21.15.189:0/1930629186' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.189", "placement": "1;smithi189=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:37:57.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[29446]: Saving service iscsi.datapool spec with placement smithi189=iscsi.a;count:1 2024-09-11T08:37:57.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:57.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:57.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:37:57.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:37:57.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:37:57.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:57 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:37:58.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:58 smithi189 ceph-mon[32675]: Deploying daemon iscsi.iscsi.a on smithi189 2024-09-11T08:37:58.241 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:37:58 smithi189 systemd[1]: Starting Ceph iscsi.iscsi.a for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:37:58.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:58 smithi131 ceph-mon[37208]: Deploying daemon iscsi.iscsi.a on smithi189 2024-09-11T08:37:58.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:58 smithi131 ceph-mon[29446]: Deploying daemon iscsi.iscsi.a on smithi189 2024-09-11T08:37:59.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:59 smithi189 ceph-mon[32675]: pgmap v180: 132 pgs: 1 creating+activating, 6 creating+peering, 12 unknown, 113 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-09-11T08:37:59.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:37:59 smithi189 ceph-mon[32675]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:37:59.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:59 smithi131 ceph-mon[37208]: pgmap v180: 132 pgs: 1 creating+activating, 6 creating+peering, 12 unknown, 113 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-09-11T08:37:59.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:37:59 smithi131 ceph-mon[37208]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:37:59.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:59 smithi131 ceph-mon[29446]: pgmap v180: 132 pgs: 1 creating+activating, 6 creating+peering, 12 unknown, 113 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-09-11T08:37:59.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:37:59 smithi131 ceph-mon[29446]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:37:59.395 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled prometheus update... 2024-09-11T08:37:59.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:37:59 smithi189 podman[59566]: 2024-09-11 08:37:59.265101586 +0000 UTC m=+0.247038219 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:37:59.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:37:59 smithi189 podman[59566]: 2024-09-11 08:37:59.375928683 +0000 UTC m=+0.357865314 container create c57800cb1cda8ef7be4b7dc897b899bfa5d7cc9bb640ff540d7b489a5176acac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:38:00.008 DEBUG:teuthology.orchestra.run.smithi189:prometheus.a> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-11T08:38:00.011 INFO:tasks.cephadm:Adding node-exporter.a on smithi131 2024-09-11T08:38:00.011 INFO:tasks.cephadm:Adding node-exporter.b on smithi189 2024-09-11T08:38:00.011 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi131=a;smithi189=b' 2024-09-11T08:38:00.240 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:00 smithi189 podman[59665]: 2024-09-11 08:38:00.031034783 +0000 UTC m=+0.017149915 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:38:00.240 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:00 smithi189 podman[59665]: 2024-09-11 08:38:00.142517463 +0000 UTC m=+0.128632583 container create c06b1ffe67470c3e97d216cc2fec2d56e01d11c3b21ee5fc0fed0119f87a1652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:38:00.500 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:00 smithi189 ceph-mon[32675]: from='client.24436 v1:172.21.15.189:0/3255629862' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:00.500 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:00 smithi189 ceph-mon[32675]: Saving service prometheus spec with placement smithi189=a;count:1 2024-09-11T08:38:00.500 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:00 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:00.500 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:00 smithi189 ceph-mon[32675]: pgmap v182: 132 pgs: 1 creating+activating, 131 active+clean; 580 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 3.9 KiB/s wr, 106 op/s 2024-09-11T08:38:00.500 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:00 smithi189 podman[59566]: 2024-09-11 08:38:00.345250628 +0000 UTC m=+1.327187257 container init c57800cb1cda8ef7be4b7dc897b899bfa5d7cc9bb640ff540d7b489a5176acac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True) 2024-09-11T08:38:00.501 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:00 smithi189 podman[59566]: 2024-09-11 08:38:00.3486869 +0000 UTC m=+1.330623532 container start c57800cb1cda8ef7be4b7dc897b899bfa5d7cc9bb640ff540d7b489a5176acac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-11T08:38:00.501 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:00 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu[59659]: log file path now is '/var/log/tcmu-runner.log' 2024-09-11T08:38:00.501 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:00 smithi189 bash[59566]: c57800cb1cda8ef7be4b7dc897b899bfa5d7cc9bb640ff540d7b489a5176acac 2024-09-11T08:38:00.865 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:38:00.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:00 smithi131 ceph-mon[37208]: from='client.24436 v1:172.21.15.189:0/3255629862' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:00.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:00 smithi131 ceph-mon[37208]: Saving service prometheus spec with placement smithi189=a;count:1 2024-09-11T08:38:00.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:00 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:00.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:00 smithi131 ceph-mon[37208]: pgmap v182: 132 pgs: 1 creating+activating, 131 active+clean; 580 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 3.9 KiB/s wr, 106 op/s 2024-09-11T08:38:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:00 smithi131 ceph-mon[29446]: from='client.24436 v1:172.21.15.189:0/3255629862' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:00 smithi131 ceph-mon[29446]: Saving service prometheus spec with placement smithi189=a;count:1 2024-09-11T08:38:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:00 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:00 smithi131 ceph-mon[29446]: pgmap v182: 132 pgs: 1 creating+activating, 131 active+clean; 580 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 3.9 KiB/s wr, 106 op/s 2024-09-11T08:38:00.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:00 smithi189 podman[59665]: 2024-09-11 08:38:00.696426743 +0000 UTC m=+0.682541862 container init c06b1ffe67470c3e97d216cc2fec2d56e01d11c3b21ee5fc0fed0119f87a1652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:38:00.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:00 smithi189 podman[59665]: 2024-09-11 08:38:00.699845075 +0000 UTC m=+0.685960194 container start c06b1ffe67470c3e97d216cc2fec2d56e01d11c3b21ee5fc0fed0119f87a1652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T08:38:00.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:00 smithi189 bash[59665]: c06b1ffe67470c3e97d216cc2fec2d56e01d11c3b21ee5fc0fed0119f87a1652 2024-09-11T08:38:00.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:00 smithi189 systemd[1]: Started Ceph iscsi.iscsi.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:38:01.271 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:00 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug Started the configuration object watcher 2024-09-11T08:38:01.271 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:00 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug Checking for config object changes every 1s 2024-09-11T08:38:01.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:00 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug Processing osd blocklist entries for this node 2024-09-11T08:38:01.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug Reading the configuration object to update local LIO configuration 2024-09-11T08:38:01.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug Configuration does not have an entry for this host(smithi189.front.sepia.ceph.com) - nothing to define to LIO 2024-09-11T08:38:01.686 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-11T08:38:01.686 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: * Environment: production 2024-09-11T08:38:01.686 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:38:01.686 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: Use a production WSGI server instead. 2024-09-11T08:38:01.687 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: * Debug mode: off 2024-09-11T08:38:01.687 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug * Running on all addresses. 2024-09-11T08:38:01.687 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:38:01.687 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: * Running on all addresses. 2024-09-11T08:38:01.687 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:38:01.687 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug * Running on http://[fe80::ec4:7aff:fe88:6df1]:5000/ (Press CTRL+C to quit) 2024-09-11T08:38:01.687 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: * Running on http://[fe80::ec4:7aff:fe88:6df1]:5000/ (Press CTRL+C to quit) 2024-09-11T08:38:01.962 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:01.962 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:01.962 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:01.962 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-mon[32675]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:38:01.962 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:01.962 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:01.962 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:01 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.189:0/2150721080' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:38:02.125 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:01 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:02.125 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:01 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:02.125 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:01 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:02.125 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:01 smithi131 ceph-mon[37208]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:38:02.125 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:01 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:02.125 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:01 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:02.125 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:01 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.189:0/2150721080' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:38:02.125 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:01 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:02.126 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:01 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:02.126 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:01 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:02.126 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:01 smithi131 ceph-mon[29446]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:38:02.126 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:01 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:02.126 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:01 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:02.126 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:01 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.189:0/2150721080' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:38:02.665 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled node-exporter update... 2024-09-11T08:38:02.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:02 smithi189 ceph-mon[32675]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s rd, 5.6 KiB/s wr, 174 op/s 2024-09-11T08:38:02.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:02 smithi189 ceph-mon[32675]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:38:02.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:02 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:03.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:02 smithi131 ceph-mon[37208]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s rd, 5.6 KiB/s wr, 174 op/s 2024-09-11T08:38:03.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:02 smithi131 ceph-mon[37208]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:38:03.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:02 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:02 smithi131 ceph-mon[29446]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s rd, 5.6 KiB/s wr, 174 op/s 2024-09-11T08:38:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:02 smithi131 ceph-mon[29446]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:38:03.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:02 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:03.797 DEBUG:teuthology.orchestra.run.smithi131:node-exporter.a> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-11T08:38:03.801 DEBUG:teuthology.orchestra.run.smithi189:node-exporter.b> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-11T08:38:03.803 INFO:tasks.cephadm:Adding alertmanager.a on smithi131 2024-09-11T08:38:03.803 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi131=a' 2024-09-11T08:38:04.105 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:38:04.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:03 smithi131 ceph-mon[37208]: from='client.24472 v1:172.21.15.189:0/1165369396' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi131=a;smithi189=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:04.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:03 smithi131 ceph-mon[37208]: Saving service node-exporter spec with placement smithi131=a;smithi189=b;count:2 2024-09-11T08:38:04.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:03 smithi131 ceph-mon[29446]: from='client.24472 v1:172.21.15.189:0/1165369396' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi131=a;smithi189=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:04.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:03 smithi131 ceph-mon[29446]: Saving service node-exporter spec with placement smithi131=a;smithi189=b;count:2 2024-09-11T08:38:04.224 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:03 smithi189 ceph-mon[32675]: from='client.24472 v1:172.21.15.189:0/1165369396' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi131=a;smithi189=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:04.224 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:03 smithi189 ceph-mon[32675]: Saving service node-exporter spec with placement smithi131=a;smithi189=b;count:2 2024-09-11T08:38:05.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:04 smithi189 ceph-mon[32675]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 139 op/s 2024-09-11T08:38:05.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:04 smithi131 ceph-mon[37208]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 139 op/s 2024-09-11T08:38:05.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:04 smithi131 ceph-mon[29446]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 139 op/s 2024-09-11T08:38:06.094 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled alertmanager update... 2024-09-11T08:38:06.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:06 smithi189 ceph-mon[32675]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 3.7 KiB/s wr, 119 op/s 2024-09-11T08:38:06.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:06 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:06.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:06 smithi189 ceph-mon[32675]: from='client.24475 v1:172.21.15.189:0/1612272624' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi131=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:06.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:06 smithi189 ceph-mon[32675]: Saving service alertmanager spec with placement smithi131=a;count:1 2024-09-11T08:38:06.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:06 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:06.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:06 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:06.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:06 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:06.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:06 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:06.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:06 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:06.650 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:06 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:06.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[29446]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 3.7 KiB/s wr, 119 op/s 2024-09-11T08:38:06.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:06.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[29446]: from='client.24475 v1:172.21.15.189:0/1612272624' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi131=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:06.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[29446]: Saving service alertmanager spec with placement smithi131=a;count:1 2024-09-11T08:38:06.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:06.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:06.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:06.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:06.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:06.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:06.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[37208]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 3.7 KiB/s wr, 119 op/s 2024-09-11T08:38:06.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:06.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[37208]: from='client.24475 v1:172.21.15.189:0/1612272624' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi131=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:06.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[37208]: Saving service alertmanager spec with placement smithi131=a;count:1 2024-09-11T08:38:06.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:06.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:06.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:06.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:06.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:06.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:06 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:06.932 DEBUG:teuthology.orchestra.run.smithi131:alertmanager.a> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-11T08:38:06.935 INFO:tasks.cephadm:Adding grafana.a on smithi189 2024-09-11T08:38:06.935 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi189=a' 2024-09-11T08:38:07.197 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:38:07.584 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:07 smithi189 ceph-mon[32675]: Deploying daemon prometheus.a on smithi189 2024-09-11T08:38:07.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:07 smithi131 ceph-mon[37208]: Deploying daemon prometheus.a on smithi189 2024-09-11T08:38:07.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:07 smithi131 ceph-mon[29446]: Deploying daemon prometheus.a on smithi189 2024-09-11T08:38:08.780 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:08 smithi189 ceph-mon[32675]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 46 KiB/s rd, 3.1 KiB/s wr, 110 op/s 2024-09-11T08:38:08.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:08 smithi131 ceph-mon[29446]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 46 KiB/s rd, 3.1 KiB/s wr, 110 op/s 2024-09-11T08:38:08.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:08 smithi131 ceph-mon[37208]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 46 KiB/s rd, 3.1 KiB/s wr, 110 op/s 2024-09-11T08:38:09.999 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled grafana update... 2024-09-11T08:38:11.012 DEBUG:teuthology.orchestra.run.smithi189:grafana.a> sudo journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@grafana.a.service 2024-09-11T08:38:11.014 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T08:38:11.014 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T08:38:11.240 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:38:11.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:11 smithi189 ceph-mon[32675]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 30 KiB/s rd, 2.1 KiB/s wr, 73 op/s 2024-09-11T08:38:11.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:11 smithi189 ceph-mon[32675]: from='client.24481 v1:172.21.15.189:0/1022467918' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:11.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:11 smithi189 ceph-mon[32675]: Saving service grafana spec with placement smithi189=a;count:1 2024-09-11T08:38:11.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:11 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:11.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:11 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:11.267 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:11.388 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:11 smithi131 ceph-mon[37208]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 30 KiB/s rd, 2.1 KiB/s wr, 73 op/s 2024-09-11T08:38:11.388 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:11 smithi131 ceph-mon[37208]: from='client.24481 v1:172.21.15.189:0/1022467918' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:11.388 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:11 smithi131 ceph-mon[37208]: Saving service grafana spec with placement smithi189=a;count:1 2024-09-11T08:38:11.389 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:11 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:11.389 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:11 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:11.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:11 smithi131 ceph-mon[29446]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 30 KiB/s rd, 2.1 KiB/s wr, 73 op/s 2024-09-11T08:38:11.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:11 smithi131 ceph-mon[29446]: from='client.24481 v1:172.21.15.189:0/1022467918' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi189=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:38:11.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:11 smithi131 ceph-mon[29446]: Saving service grafana spec with placement smithi189=a;count:1 2024-09-11T08:38:11.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:11 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:11.389 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:11 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:12.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:12 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:12.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:12 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:12.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:12 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:13.252 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-09-11T08:38:13.252 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD1VuFm+GKPDhAARkKQR+ig0xvODdMm9Lz5kA== 2024-09-11T08:38:13.262 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:13 smithi131 ceph-mon[37208]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 1.3 KiB/s wr, 46 op/s 2024-09-11T08:38:13.262 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:13 smithi131 ceph-mon[29446]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 1.3 KiB/s wr, 46 op/s 2024-09-11T08:38:13.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:13 smithi189 ceph-mon[32675]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 1.3 KiB/s wr, 46 op/s 2024-09-11T08:38:13.894 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-11T08:38:13.894 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T08:38:13.894 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T08:38:13.932 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T08:38:14.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:14 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3652630304' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:38:14.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:14 smithi131 ceph-mon[29446]: from='client.24482 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:38:14.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:14 smithi131 ceph-mon[29446]: from='client.24482 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:38:14.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:14 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3652630304' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:38:14.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:14 smithi131 ceph-mon[37208]: from='client.24482 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:38:14.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:14 smithi131 ceph-mon[37208]: from='client.24482 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:38:14.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:14 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3652630304' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:38:14.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:14 smithi189 ceph-mon[32675]: from='client.24482 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:38:14.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:14 smithi189 ceph-mon[32675]: from='client.24482 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:38:15.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:15 smithi189 ceph-mon[32675]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:38:15.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:15 smithi131 ceph-mon[37208]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:38:15.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:15 smithi131 ceph-mon[29446]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:38:16.327 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:38:17.301 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:17 smithi189 ceph-mon[32675]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:38:17.301 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:17 smithi189 systemd[1]: Starting Ceph prometheus.a for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:38:17.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:17 smithi131 ceph-mon[29446]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:38:17.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:17 smithi131 ceph-mon[37208]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:38:19.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:19 smithi131 ceph-mon[37208]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:38:19.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:19 smithi131 ceph-mon[29446]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:38:19.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:19 smithi189 ceph-mon[32675]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:38:20.101 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 podman[61292]: 2024-09-11 08:38:19.93063145 +0000 UTC m=+0.748512716 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-11T08:38:20.345 INFO:teuthology.orchestra.run.smithi189.stdout:[client.1] 2024-09-11T08:38:20.345 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQD8VuFmyclYFBAA90/9ycTrcMwkXt+oJELlyA== 2024-09-11T08:38:20.364 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 podman[61292]: 2024-09-11 08:38:20.244608951 +0000 UTC m=+1.062490219 container create d75fb03a78480de74e2ccec511804f08fa233a2ed18d9e60e36180ec7d1a4812 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:38:20.741 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 podman[61292]: 2024-09-11 08:38:20.472387191 +0000 UTC m=+1.290268457 container init d75fb03a78480de74e2ccec511804f08fa233a2ed18d9e60e36180ec7d1a4812 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:38:20.741 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 podman[61292]: 2024-09-11 08:38:20.475363238 +0000 UTC m=+1.293244505 container start d75fb03a78480de74e2ccec511804f08fa233a2ed18d9e60e36180ec7d1a4812 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:38:20.741 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.499Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T08:38:20.741 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.499Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T08:38:20.741 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.499Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi189 (none))" 2024-09-11T08:38:20.741 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.499Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T08:38:20.741 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.499Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T08:38:20.742 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.500Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T08:38:20.742 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.501Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T08:38:20.742 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.502Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T08:38:20.742 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.502Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T08:38:20.742 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.504Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T08:38:20.742 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.504Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.887µs 2024-09-11T08:38:20.742 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.504Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T08:38:20.742 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.504Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-11T08:38:20.742 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.504Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=18.661µs wal_replay_duration=265.844µs wbl_replay_duration=79ns total_replay_duration=303.966µs 2024-09-11T08:38:20.742 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.505Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T08:38:20.742 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.505Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T08:38:20.742 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.505Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T08:38:20.743 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.520Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.608231ms db_storage=673ns remote_storage=1.071µs web_handler=311ns query_engine=503ns scrape=447.774µs scrape_sd=15.712µs notify=653ns notify_sd=1.262µs rules=13.914915ms tracing=6.517µs 2024-09-11T08:38:20.743 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.520Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T08:38:20.743 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:38:20.520Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T08:38:20.743 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 bash[61292]: d75fb03a78480de74e2ccec511804f08fa233a2ed18d9e60e36180ec7d1a4812 2024-09-11T08:38:20.743 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:38:20 smithi189 systemd[1]: Started Ceph prometheus.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:38:21.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:21 smithi189 ceph-mon[32675]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:38:21.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:21 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.189:0/792476036' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:38:21.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:21 smithi189 ceph-mon[32675]: from='client.24493 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:38:21.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:21 smithi189 ceph-mon[32675]: from='client.24493 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:38:21.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:21 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:21.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:21 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:21.202 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:21 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:21.202 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:21 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:38:21.202 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:21 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:38:21.224 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-11T08:38:21.224 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-11T08:38:21.225 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-11T08:38:21.263 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T08:38:21.263 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T08:38:21.263 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T08:38:21.290 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-mon[37208]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:38:21.290 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.189:0/792476036' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:38:21.290 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-mon[37208]: from='client.24493 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:38:21.290 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-mon[37208]: from='client.24493 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:38:21.291 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:21.291 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:21.291 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:21.291 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:38:21.291 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-mon[29446]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:38:21.291 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.189:0/792476036' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:38:21.291 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-mon[29446]: from='client.24493 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:38:21.291 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-mon[29446]: from='client.24493 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:38:21.291 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:21.291 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:21.291 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' 2024-09-11T08:38:21.291 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:38:21.515 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:21.891 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ignoring --setuser ceph since I am not root 2024-09-11T08:38:21.891 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ignoring --setgroup ceph since I am not root 2024-09-11T08:38:21.891 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:21.797+0000 7f667f471ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:38:21.891 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:21.854+0000 7f667f471ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:38:21.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:21 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ignoring --setuser ceph since I am not root 2024-09-11T08:38:21.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:21 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ignoring --setgroup ceph since I am not root 2024-09-11T08:38:21.991 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:21 smithi189 ceph-mgr[34509]: -- 172.21.15.189:0/2326832097 <== mon.1 v1:172.21.15.189:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 4217036246 0 0) 0x55a06793c820 con 0x55a067948c00 2024-09-11T08:38:21.991 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:21 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:21.792+0000 7fa00c71aec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:38:21.991 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:21 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:21.846+0000 7fa00c71aec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:38:21.991 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:21 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:21.910+0000 7fa00c71aec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:38:22.188 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:22 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:22.188 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:22 smithi131 ceph-mon[37208]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:38:22.188 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:22 smithi131 ceph-mon[37208]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:38:22.189 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:21 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:21.927+0000 7f667f471ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:38:22.189 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:22 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:22.052+0000 7f667f471ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:38:22.189 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:22 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:22.189 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:22 smithi131 ceph-mon[29446]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:38:22.189 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:22 smithi131 ceph-mon[29446]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:38:22.424 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:22 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:22.424 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:22 smithi189 ceph-mon[32675]: from='mgr.14150 v1:172.21.15.131:0/4164404583' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:38:22.424 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:22 smithi189 ceph-mon[32675]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:38:22.424 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:22 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:22.034+0000 7fa00c71aec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:38:22.424 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:22 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:22.168+0000 7fa00c71aec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:38:22.425 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:22 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:22.424+0000 7fa00c71aec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:38:22.452 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:22 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:22.188+0000 7f667f471ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:38:22.729 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:22 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:22.451+0000 7f667f471ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:38:22.729 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:22 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:22.512+0000 7f667f471ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:38:22.729 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:22 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:22.676+0000 7f667f471ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:38:22.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:22 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:22.483+0000 7fa00c71aec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:38:22.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:22 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:22.643+0000 7fa00c71aec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:38:23.487 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:38:23.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:23 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2710460185' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:38:23.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:23.481+0000 7fa00c71aec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:38:23.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:23.545+0000 7fa00c71aec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:38:23.835 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:23 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2710460185' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:38:23.836 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:23 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2710460185' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:38:23.836 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:23 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:23.573+0000 7f667f471ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:38:23.836 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:23 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:23.638+0000 7f667f471ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:38:24.278 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6800","nonce":1346976293}]},"active_addr":"172.21.15.131:6800/1346976293","active_change":"2024-09-11T08:33:35.432916+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24109,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.131:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":3775471527}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":2746482984}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":3734879262}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":377420170}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":362018701}]}]} 2024-09-11T08:38:24.283 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T08:38:24.283 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T08:38:24.283 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:38:24.528 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:24.241+0000 7fa00c71aec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:38:24.529 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:24.352+0000 7fa00c71aec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:38:24.529 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:24.470+0000 7fa00c71aec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:38:24.585 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:24 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:24.366+0000 7f667f471ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:38:24.585 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:24 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:24.483+0000 7f667f471ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:38:24.585 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:24.836 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:24 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:24.626+0000 7f667f471ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:38:24.836 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:24 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:24.685+0000 7f667f471ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:38:24.836 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:24 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:24.815+0000 7f667f471ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:38:24.911 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:24.528+0000 7fa00c71aec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:38:24.911 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:24.653+0000 7fa00c71aec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:38:24.911 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:24.720+0000 7fa00c71aec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:38:24.911 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:24.777+0000 7fa00c71aec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:38:24.911 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:24.911+0000 7fa00c71aec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:38:25.092 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:24 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:24.886+0000 7f667f471ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:38:25.120 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:24 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:24.943+0000 7f667f471ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:38:25.120 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:25 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:25.082+0000 7f667f471ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:38:25.178 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:24.974+0000 7fa00c71aec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:38:25.178 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:25 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:25.038+0000 7fa00c71aec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:38:25.178 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:25 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:25.120+0000 7fa00c71aec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:38:25.178 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:25 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:25.177+0000 7fa00c71aec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:38:25.350 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:25 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:25.146+0000 7f667f471ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:38:25.351 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:25 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:25.212+0000 7f667f471ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:38:25.351 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:25 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:25.293+0000 7f667f471ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:38:25.351 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:25 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:25.350+0000 7f667f471ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:38:25.490 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:25 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:25.235+0000 7fa00c71aec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:38:25.640 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:25 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:25.409+0000 7f667f471ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:38:26.115 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-mon[29446]: Standby manager daemon x restarted 2024-09-11T08:38:26.116 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-mon[29446]: Standby manager daemon x started 2024-09-11T08:38:26.116 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-mon[29446]: from='mgr.? v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:38:26.116 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-mon[29446]: from='mgr.? v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:38:26.116 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-mon[29446]: from='mgr.? v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:38:26.116 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-mon[29446]: from='mgr.? v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:38:26.116 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-mon[37208]: Standby manager daemon x restarted 2024-09-11T08:38:26.116 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-mon[37208]: Standby manager daemon x started 2024-09-11T08:38:26.116 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-mon[37208]: from='mgr.? v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:38:26.116 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-mon[37208]: from='mgr.? v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:38:26.116 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-mon[37208]: from='mgr.? v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:38:26.117 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-mon[37208]: from='mgr.? v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:38:26.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:26 smithi189 ceph-mon[32675]: Standby manager daemon x restarted 2024-09-11T08:38:26.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:26 smithi189 ceph-mon[32675]: Standby manager daemon x started 2024-09-11T08:38:26.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:26 smithi189 ceph-mon[32675]: from='mgr.? v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:38:26.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:26 smithi189 ceph-mon[32675]: from='mgr.? v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:38:26.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:26 smithi189 ceph-mon[32675]: from='mgr.? v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:38:26.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:26 smithi189 ceph-mon[32675]: from='mgr.? v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:38:26.241 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:25 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:25.971+0000 7fa00c71aec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:38:26.241 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:25 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: [11/Sep/2024:08:38:25] ENGINE Bus STARTING 2024-09-11T08:38:26.241 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:25 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: CherryPy Checker: 2024-09-11T08:38:26.241 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:25 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: The Application mounted at '' has an empty config. 2024-09-11T08:38:26.241 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:25 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:38:26.241 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:26 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: [11/Sep/2024:08:38:26] ENGINE Serving on http://:::9283 2024-09-11T08:38:26.241 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:26 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: [11/Sep/2024:08:38:26] ENGINE Bus STARTED 2024-09-11T08:38:26.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:26.185+0000 7f667f471ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:38:26.391 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:38:26] ENGINE Bus STARTING 2024-09-11T08:38:26.391 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: CherryPy Checker: 2024-09-11T08:38:26.391 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: The Application mounted at '' has an empty config. 2024-09-11T08:38:26.391 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:26.539 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:38:26.539 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":61,"fsid":"50e4d0bc-7018-11ef-bcea-c7b262605968","created":"2024-09-11T08:32:52.578895+0000","modified":"2024-09-11T08:38:26.187325+0000","last_up_change":"2024-09-11T08:37:37.187181+0000","last_in_change":"2024-09-11T08:37:20.495590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:35:43.725581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-11T08:37:47.417642+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-11T08:37:47.794519+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T08:37:49.292477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T08:37:51.271812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T08:37:53.344575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"2a4b0f7e-5c59-472b-ac6b-fefc3f09e091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6801","nonce":1075394441}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6802","nonce":1075394441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6804","nonce":1075394441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6803","nonce":1075394441}]},"public_addr":"172.21.15.131:6801/1075394441","cluster_addr":"172.21.15.131:6802/1075394441","heartbeat_back_addr":"172.21.15.131:6804/1075394441","heartbeat_front_addr":"172.21.15.131:6803/1075394441","state":["exists","up"]},{"osd":1,"uuid":"6747dbcc-3002-4d59-8546-f28ad65bc25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6805","nonce":2577658258}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6806","nonce":2577658258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6808","nonce":2577658258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6807","nonce":2577658258}]},"public_addr":"172.21.15.131:6805/2577658258","cluster_addr":"172.21.15.131:6806/2577658258","heartbeat_back_addr":"172.21.15.131:6808/2577658258","heartbeat_front_addr":"172.21.15.131:6807/2577658258","state":["exists","up"]},{"osd":2,"uuid":"e63d564e-1ffc-4575-a882-89a1342664ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6809","nonce":4000463570}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6810","nonce":4000463570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6812","nonce":4000463570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6811","nonce":4000463570}]},"public_addr":"172.21.15.131:6809/4000463570","cluster_addr":"172.21.15.131:6810/4000463570","heartbeat_back_addr":"172.21.15.131:6812/4000463570","heartbeat_front_addr":"172.21.15.131:6811/4000463570","state":["exists","up"]},{"osd":3,"uuid":"b5a1a11f-ca8a-40ea-a94b-9c6d71ab7565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6813","nonce":2422737852}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6814","nonce":2422737852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6816","nonce":2422737852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6815","nonce":2422737852}]},"public_addr":"172.21.15.131:6813/2422737852","cluster_addr":"172.21.15.131:6814/2422737852","heartbeat_back_addr":"172.21.15.131:6816/2422737852","heartbeat_front_addr":"172.21.15.131:6815/2422737852","state":["exists","up"]},{"osd":4,"uuid":"f1b6e461-b362-4c98-bdac-0e5babb0849e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6800","nonce":2575698578}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6801","nonce":2575698578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6803","nonce":2575698578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6802","nonce":2575698578}]},"public_addr":"172.21.15.189:6800/2575698578","cluster_addr":"172.21.15.189:6801/2575698578","heartbeat_back_addr":"172.21.15.189:6803/2575698578","heartbeat_front_addr":"172.21.15.189:6802/2575698578","state":["exists","up"]},{"osd":5,"uuid":"114e4e0e-dc20-454e-8456-ac3f850abed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6804","nonce":571798110}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6805","nonce":571798110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6807","nonce":571798110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6806","nonce":571798110}]},"public_addr":"172.21.15.189:6804/571798110","cluster_addr":"172.21.15.189:6805/571798110","heartbeat_back_addr":"172.21.15.189:6807/571798110","heartbeat_front_addr":"172.21.15.189:6806/571798110","state":["exists","up"]},{"osd":6,"uuid":"f5d731e5-cfb5-4752-a8da-6fe2ae517b5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6808","nonce":1725667081}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6809","nonce":1725667081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6811","nonce":1725667081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6810","nonce":1725667081}]},"public_addr":"172.21.15.189:6808/1725667081","cluster_addr":"172.21.15.189:6809/1725667081","heartbeat_back_addr":"172.21.15.189:6811/1725667081","heartbeat_front_addr":"172.21.15.189:6810/1725667081","state":["exists","up"]},{"osd":7,"uuid":"99012398-2a41-475d-918c-421d581f11a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6812","nonce":1579379153}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6813","nonce":1579379153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6815","nonce":1579379153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6814","nonce":1579379153}]},"public_addr":"172.21.15.189:6812/1579379153","cluster_addr":"172.21.15.189:6813/1579379153","heartbeat_back_addr":"172.21.15.189:6815/1579379153","heartbeat_front_addr":"172.21.15.189:6814/1579379153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:34:52.816843+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:35:17.202169+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:35:41.237351+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:36:05.630435+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:36:26.230762+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:36:49.043341+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:37:12.486687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:37:35.285916+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3775471527":"2024-09-12T08:38:26.187261+0000","172.21.15.131:0/2726656335":"2024-09-12T08:33:13.813660+0000","172.21.15.131:0/1761772131":"2024-09-12T08:33:13.813660+0000","172.21.15.131:6800/1673105920":"2024-09-12T08:33:35.432760+0000","172.21.15.131:0/2746482984":"2024-09-12T08:38:26.187261+0000","172.21.15.131:0/2175368038":"2024-09-12T08:33:35.432760+0000","172.21.15.131:0/957761929":"2024-09-12T08:33:35.432760+0000","172.21.15.131:0/1949349877":"2024-09-12T08:33:35.432760+0000","172.21.15.131:6800/3956075890":"2024-09-12T08:33:13.813660+0000","172.21.15.131:0/470181260":"2024-09-12T08:33:13.813660+0000","172.21.15.131:6800/1346976293":"2024-09-12T08:38:26.187261+0000","172.21.15.131:0/377420170":"2024-09-12T08:38:26.187261+0000","172.21.15.131:0/3734879262":"2024-09-12T08:38:26.187261+0000","172.21.15.131:0/362018701":"2024-09-12T08:38:26.187261+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:38:26.891 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:38:26] ENGINE Serving on http://:::9283 2024-09-11T08:38:26.891 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:38:26] ENGINE Bus STARTED 2024-09-11T08:38:27.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-11T08:38:27.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: Active manager daemon y restarted 2024-09-11T08:38:27.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: Activating manager daemon y 2024-09-11T08:38:27.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:38:27.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: mgrmap e19: y(active, starting, since 0.013084s), standbys: x 2024-09-11T08:38:27.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:38:27.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:38:27.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:38:27.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:38:27.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:38:27.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:38:27.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:38:27.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:38:27.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:38:27.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:38:27.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:38:27.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:38:27.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:27.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:38:27.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:38:27.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:38:27.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: Manager daemon y is now available 2024-09-11T08:38:27.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:27.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:27.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:27.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:38:27.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:38:27.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:38:27.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:38:27.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2524163006' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:38:27.275 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-11T08:38:27.275 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: Active manager daemon y restarted 2024-09-11T08:38:27.275 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: Activating manager daemon y 2024-09-11T08:38:27.275 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:38:27.276 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: mgrmap e19: y(active, starting, since 0.013084s), standbys: x 2024-09-11T08:38:27.276 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:38:27.276 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:38:27.276 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:38:27.276 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:38:27.276 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:38:27.276 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:38:27.276 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:38:27.276 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:38:27.276 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:38:27.276 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:38:27.276 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:38:27.276 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:38:27.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:27.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:38:27.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:38:27.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:38:27.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: Manager daemon y is now available 2024-09-11T08:38:27.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:27.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:27.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:27.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:38:27.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:38:27.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:38:27.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:38:27.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2524163006' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:38:27.296 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T08:38:27.297 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:38:27.320 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-11T08:38:27.320 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: Active manager daemon y restarted 2024-09-11T08:38:27.320 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: Activating manager daemon y 2024-09-11T08:38:27.320 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:38:27.320 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: mgrmap e19: y(active, starting, since 0.013084s), standbys: x 2024-09-11T08:38:27.320 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:38:27.320 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:38:27.320 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:38:27.320 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:38:27.320 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:38:27.321 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:38:27.321 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:38:27.321 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:38:27.321 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:38:27.321 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:38:27.321 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:38:27.321 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:38:27.321 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:38:27.321 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:38:27.321 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:38:27.321 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:38:27.321 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: Manager daemon y is now available 2024-09-11T08:38:27.321 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:38:27.321 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:27.322 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:27.322 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:38:27.322 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:38:27.322 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:38:27.322 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:38:27.322 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:27 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2524163006' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:38:27.876 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:27.772+0000 7f6622233640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:38:27] ENGINE Error in HTTPServer.serve 2024-09-11T08:38:27.877 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: Traceback (most recent call last): 2024-09-11T08:38:27.877 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:38:27.877 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: self._connections.run(self.expiration_interval) 2024-09-11T08:38:27.877 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:38:27.877 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: self._run(expiration_interval) 2024-09-11T08:38:27.877 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:38:27.877 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:38:27.877 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:38:27.877 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:38:27.877 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:38:27.877 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: s = self.context.wrap_socket( 2024-09-11T08:38:27.877 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:38:27.878 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: return self.sslsocket_class._create( 2024-09-11T08:38:27.878 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:38:27.878 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: self.do_handshake() 2024-09-11T08:38:27.878 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:38:27.878 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: self._sslobj.do_handshake() 2024-09-11T08:38:27.878 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:38:27.878 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:38:27 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:38:28.147 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:28.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:28 smithi189 ceph-mon[32675]: mgrmap e20: y(active, since 1.01438s), standbys: x 2024-09-11T08:38:28.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:28 smithi189 ceph-mon[32675]: [11/Sep/2024:08:38:27] ENGINE Bus STARTING 2024-09-11T08:38:28.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:28 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:28.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:28 smithi131 ceph-mon[37208]: mgrmap e20: y(active, since 1.01438s), standbys: x 2024-09-11T08:38:28.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:28 smithi131 ceph-mon[37208]: [11/Sep/2024:08:38:27] ENGINE Bus STARTING 2024-09-11T08:38:28.497 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:28 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:28.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:28 smithi131 ceph-mon[29446]: mgrmap e20: y(active, since 1.01438s), standbys: x 2024-09-11T08:38:28.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:28 smithi131 ceph-mon[29446]: [11/Sep/2024:08:38:27] ENGINE Bus STARTING 2024-09-11T08:38:28.497 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:28 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:29.469 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:29 smithi131 ceph-mon[37208]: [11/Sep/2024:08:38:27] ENGINE Serving on https://172.21.15.131:7150 2024-09-11T08:38:29.469 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:29 smithi131 ceph-mon[37208]: [11/Sep/2024:08:38:27] ENGINE Bus STARTED 2024-09-11T08:38:29.469 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:29 smithi131 ceph-mon[37208]: [11/Sep/2024:08:38:27] ENGINE Error in HTTPServer.serve 2024-09-11T08:38:29.469 INFO:journalctl@ceph.mon.c.smithi131.stdout: Traceback (most recent call last): 2024-09-11T08:38:29.469 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:38:29.469 INFO:journalctl@ceph.mon.c.smithi131.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:38:29.469 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:38:29.469 INFO:journalctl@ceph.mon.c.smithi131.stdout: self._run(expiration_interval) 2024-09-11T08:38:29.469 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:38:29.469 INFO:journalctl@ceph.mon.c.smithi131.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:38:29.469 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:38:29.470 INFO:journalctl@ceph.mon.c.smithi131.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:38:29.470 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:38:29.470 INFO:journalctl@ceph.mon.c.smithi131.stdout: s = self.context.wrap_socket( 2024-09-11T08:38:29.470 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:38:29.470 INFO:journalctl@ceph.mon.c.smithi131.stdout: return self.sslsocket_class._create( 2024-09-11T08:38:29.470 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:38:29.470 INFO:journalctl@ceph.mon.c.smithi131.stdout: self.do_handshake() 2024-09-11T08:38:29.470 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:38:29.470 INFO:journalctl@ceph.mon.c.smithi131.stdout: self._sslobj.do_handshake() 2024-09-11T08:38:29.470 INFO:journalctl@ceph.mon.c.smithi131.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:38:29.470 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:29 smithi131 ceph-mon[37208]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:29.470 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:29 smithi131 ceph-mon[37208]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-11T08:38:29.470 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:29 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:29.470 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:29 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:29.471 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:29 smithi131 ceph-mon[29446]: [11/Sep/2024:08:38:27] ENGINE Serving on https://172.21.15.131:7150 2024-09-11T08:38:29.471 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:29 smithi131 ceph-mon[29446]: [11/Sep/2024:08:38:27] ENGINE Bus STARTED 2024-09-11T08:38:29.471 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:29 smithi131 ceph-mon[29446]: [11/Sep/2024:08:38:27] ENGINE Error in HTTPServer.serve 2024-09-11T08:38:29.471 INFO:journalctl@ceph.mon.a.smithi131.stdout: Traceback (most recent call last): 2024-09-11T08:38:29.471 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:38:29.471 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:38:29.471 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:38:29.471 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._run(expiration_interval) 2024-09-11T08:38:29.471 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:38:29.471 INFO:journalctl@ceph.mon.a.smithi131.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:38:29.472 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:38:29.472 INFO:journalctl@ceph.mon.a.smithi131.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:38:29.472 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:38:29.472 INFO:journalctl@ceph.mon.a.smithi131.stdout: s = self.context.wrap_socket( 2024-09-11T08:38:29.472 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:38:29.472 INFO:journalctl@ceph.mon.a.smithi131.stdout: return self.sslsocket_class._create( 2024-09-11T08:38:29.472 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:38:29.472 INFO:journalctl@ceph.mon.a.smithi131.stdout: self.do_handshake() 2024-09-11T08:38:29.472 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:38:29.472 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._sslobj.do_handshake() 2024-09-11T08:38:29.472 INFO:journalctl@ceph.mon.a.smithi131.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:38:29.472 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:29 smithi131 ceph-mon[29446]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:29.472 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:29 smithi131 ceph-mon[29446]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-11T08:38:29.472 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:29 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:29.473 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:29 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:29.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:29 smithi189 ceph-mon[32675]: [11/Sep/2024:08:38:27] ENGINE Serving on https://172.21.15.131:7150 2024-09-11T08:38:29.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:29 smithi189 ceph-mon[32675]: [11/Sep/2024:08:38:27] ENGINE Bus STARTED 2024-09-11T08:38:29.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:29 smithi189 ceph-mon[32675]: [11/Sep/2024:08:38:27] ENGINE Error in HTTPServer.serve 2024-09-11T08:38:29.490 INFO:journalctl@ceph.mon.b.smithi189.stdout: Traceback (most recent call last): 2024-09-11T08:38:29.490 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:38:29.490 INFO:journalctl@ceph.mon.b.smithi189.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:38:29.490 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:38:29.490 INFO:journalctl@ceph.mon.b.smithi189.stdout: self._run(expiration_interval) 2024-09-11T08:38:29.490 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:38:29.490 INFO:journalctl@ceph.mon.b.smithi189.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:38:29.490 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:38:29.490 INFO:journalctl@ceph.mon.b.smithi189.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:38:29.491 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:38:29.491 INFO:journalctl@ceph.mon.b.smithi189.stdout: s = self.context.wrap_socket( 2024-09-11T08:38:29.491 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:38:29.491 INFO:journalctl@ceph.mon.b.smithi189.stdout: return self.sslsocket_class._create( 2024-09-11T08:38:29.491 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:38:29.491 INFO:journalctl@ceph.mon.b.smithi189.stdout: self.do_handshake() 2024-09-11T08:38:29.491 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:38:29.491 INFO:journalctl@ceph.mon.b.smithi189.stdout: self._sslobj.do_handshake() 2024-09-11T08:38:29.491 INFO:journalctl@ceph.mon.b.smithi189.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:38:29.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:29 smithi189 ceph-mon[32675]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:29.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:29 smithi189 ceph-mon[32675]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-11T08:38:29.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:29 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:29.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:29 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:29.970 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:38:29.970 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":61,"fsid":"50e4d0bc-7018-11ef-bcea-c7b262605968","created":"2024-09-11T08:32:52.578895+0000","modified":"2024-09-11T08:38:26.187325+0000","last_up_change":"2024-09-11T08:37:37.187181+0000","last_in_change":"2024-09-11T08:37:20.495590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T08:35:43.725581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-11T08:37:47.417642+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-11T08:37:47.794519+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T08:37:49.292477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T08:37:51.271812+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T08:37:53.344575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"2a4b0f7e-5c59-472b-ac6b-fefc3f09e091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6801","nonce":1075394441}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6802","nonce":1075394441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6804","nonce":1075394441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6803","nonce":1075394441}]},"public_addr":"172.21.15.131:6801/1075394441","cluster_addr":"172.21.15.131:6802/1075394441","heartbeat_back_addr":"172.21.15.131:6804/1075394441","heartbeat_front_addr":"172.21.15.131:6803/1075394441","state":["exists","up"]},{"osd":1,"uuid":"6747dbcc-3002-4d59-8546-f28ad65bc25a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6805","nonce":2577658258}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6806","nonce":2577658258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6808","nonce":2577658258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6807","nonce":2577658258}]},"public_addr":"172.21.15.131:6805/2577658258","cluster_addr":"172.21.15.131:6806/2577658258","heartbeat_back_addr":"172.21.15.131:6808/2577658258","heartbeat_front_addr":"172.21.15.131:6807/2577658258","state":["exists","up"]},{"osd":2,"uuid":"e63d564e-1ffc-4575-a882-89a1342664ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6809","nonce":4000463570}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6810","nonce":4000463570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6812","nonce":4000463570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6811","nonce":4000463570}]},"public_addr":"172.21.15.131:6809/4000463570","cluster_addr":"172.21.15.131:6810/4000463570","heartbeat_back_addr":"172.21.15.131:6812/4000463570","heartbeat_front_addr":"172.21.15.131:6811/4000463570","state":["exists","up"]},{"osd":3,"uuid":"b5a1a11f-ca8a-40ea-a94b-9c6d71ab7565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6813","nonce":2422737852}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6814","nonce":2422737852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6816","nonce":2422737852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6815","nonce":2422737852}]},"public_addr":"172.21.15.131:6813/2422737852","cluster_addr":"172.21.15.131:6814/2422737852","heartbeat_back_addr":"172.21.15.131:6816/2422737852","heartbeat_front_addr":"172.21.15.131:6815/2422737852","state":["exists","up"]},{"osd":4,"uuid":"f1b6e461-b362-4c98-bdac-0e5babb0849e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6800","nonce":2575698578}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6801","nonce":2575698578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6803","nonce":2575698578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6802","nonce":2575698578}]},"public_addr":"172.21.15.189:6800/2575698578","cluster_addr":"172.21.15.189:6801/2575698578","heartbeat_back_addr":"172.21.15.189:6803/2575698578","heartbeat_front_addr":"172.21.15.189:6802/2575698578","state":["exists","up"]},{"osd":5,"uuid":"114e4e0e-dc20-454e-8456-ac3f850abed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6804","nonce":571798110}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6805","nonce":571798110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6807","nonce":571798110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6806","nonce":571798110}]},"public_addr":"172.21.15.189:6804/571798110","cluster_addr":"172.21.15.189:6805/571798110","heartbeat_back_addr":"172.21.15.189:6807/571798110","heartbeat_front_addr":"172.21.15.189:6806/571798110","state":["exists","up"]},{"osd":6,"uuid":"f5d731e5-cfb5-4752-a8da-6fe2ae517b5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6808","nonce":1725667081}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6809","nonce":1725667081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6811","nonce":1725667081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6810","nonce":1725667081}]},"public_addr":"172.21.15.189:6808/1725667081","cluster_addr":"172.21.15.189:6809/1725667081","heartbeat_back_addr":"172.21.15.189:6811/1725667081","heartbeat_front_addr":"172.21.15.189:6810/1725667081","state":["exists","up"]},{"osd":7,"uuid":"99012398-2a41-475d-918c-421d581f11a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6812","nonce":1579379153}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6813","nonce":1579379153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6815","nonce":1579379153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.189:6814","nonce":1579379153}]},"public_addr":"172.21.15.189:6812/1579379153","cluster_addr":"172.21.15.189:6813/1579379153","heartbeat_back_addr":"172.21.15.189:6815/1579379153","heartbeat_front_addr":"172.21.15.189:6814/1579379153","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:34:52.816843+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:35:17.202169+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:35:41.237351+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:36:05.630435+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:36:26.230762+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:36:49.043341+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:37:12.486687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:37:35.285916+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/3775471527":"2024-09-12T08:38:26.187261+0000","172.21.15.131:0/2726656335":"2024-09-12T08:33:13.813660+0000","172.21.15.131:0/1761772131":"2024-09-12T08:33:13.813660+0000","172.21.15.131:6800/1673105920":"2024-09-12T08:33:35.432760+0000","172.21.15.131:0/2746482984":"2024-09-12T08:38:26.187261+0000","172.21.15.131:0/2175368038":"2024-09-12T08:33:35.432760+0000","172.21.15.131:0/957761929":"2024-09-12T08:33:35.432760+0000","172.21.15.131:0/1949349877":"2024-09-12T08:33:35.432760+0000","172.21.15.131:6800/3956075890":"2024-09-12T08:33:13.813660+0000","172.21.15.131:0/470181260":"2024-09-12T08:33:13.813660+0000","172.21.15.131:6800/1346976293":"2024-09-12T08:38:26.187261+0000","172.21.15.131:0/377420170":"2024-09-12T08:38:26.187261+0000","172.21.15.131:0/3734879262":"2024-09-12T08:38:26.187261+0000","172.21.15.131:0/362018701":"2024-09-12T08:38:26.187261+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:38:29.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:29 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:38:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:38:30.752 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T08:38:30.753 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T08:38:30.753 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T08:38:30.754 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T08:38:30.754 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T08:38:30.755 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T08:38:30.755 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T08:38:30.755 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T08:38:31.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: Adjusting osd_memory_target on smithi189 to 3760M 2024-09-11T08:38:31.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/660233999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:38:31.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:31.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: Adjusting osd_memory_target on smithi189 to 3760M 2024-09-11T08:38:31.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.084 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.084 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/660233999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:38:31.084 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:31.084 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:30 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.240 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:38:31.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: Adjusting osd_memory_target on smithi189 to 3760M 2024-09-11T08:38:31.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/660233999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:38:31.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:38:31.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:30 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.242 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:31.286 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:31.730 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:31.730 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:31.867 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:31.872 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:31.872 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:31.960 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.960 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.960 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.960 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.960 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.960 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.960 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.960 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.960 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: Adjusting osd_memory_target on smithi131 to 3760M 2024-09-11T08:38:31.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:31.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:31.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-11T08:38:31.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-11T08:38:31.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-11T08:38:31.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:31.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:38:31.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:38:31.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:38:31.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:38:31.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:38:31.961 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.962 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:31.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: Adjusting osd_memory_target on smithi131 to 3760M 2024-09-11T08:38:31.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:31.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:31.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-11T08:38:31.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-11T08:38:31.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-11T08:38:31.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:31.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:38:31.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:38:31.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:38:31.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:38:31.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:38:31.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:31.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:31 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:32.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:32.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:32.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:32.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:32.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:32.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:32.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:32.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:32.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:38:32.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: Adjusting osd_memory_target on smithi131 to 3760M 2024-09-11T08:38:32.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:32.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:38:32.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:38:32.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-11T08:38:32.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-11T08:38:32.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-11T08:38:32.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:32.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:38:32.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:38:32.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:38:32.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:38:32.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:38:32.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:32.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:31 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:33.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:33 smithi131 ceph-mon[37208]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:38:33.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:33 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:33.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:33 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:33.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:33 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:33.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:33 smithi131 ceph-mon[37208]: Deploying daemon alertmanager.a on smithi131 2024-09-11T08:38:33.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:33 smithi131 ceph-mon[37208]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1 KiB/s rd, 1 op/s 2024-09-11T08:38:33.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:33 smithi131 ceph-mon[29446]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:38:33.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:33 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:33.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:33 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:33.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:33 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:33.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:33 smithi131 ceph-mon[29446]: Deploying daemon alertmanager.a on smithi131 2024-09-11T08:38:33.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:33 smithi131 ceph-mon[29446]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1 KiB/s rd, 1 op/s 2024-09-11T08:38:33.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:33 smithi189 ceph-mon[32675]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:38:33.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:33 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:33.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:33 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:33.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:33 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:38:33.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:33 smithi189 ceph-mon[32675]: Deploying daemon alertmanager.a on smithi131 2024-09-11T08:38:33.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:33 smithi189 ceph-mon[32675]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1 KiB/s rd, 1 op/s 2024-09-11T08:38:35.568 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:35 smithi131 ceph-mon[37208]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T08:38:35.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:35 smithi131 ceph-mon[29446]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T08:38:35.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:35 smithi189 ceph-mon[32675]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T08:38:37.365 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:37 smithi131 ceph-mon[29446]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:38:37.366 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:37 smithi131 ceph-mon[37208]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:38:37.390 INFO:teuthology.orchestra.run.smithi131.stdout:34359738415 2024-09-11T08:38:37.390 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T08:38:37.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:37 smithi189 ceph-mon[32675]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:38:39.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:39 smithi131 ceph-mon[37208]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:38:39.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:39 smithi131 ceph-mon[29446]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:38:39.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:39 smithi189 ceph-mon[32675]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:38:39.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:39 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:38:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:38:41.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:41 smithi189 ceph-mon[32675]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:38:41.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:41 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:41.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:41 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:38:41.603 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:41 smithi131 ceph-mon[37208]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:38:41.603 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:41 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:41.604 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:41 smithi131 ceph-mon[29446]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:38:41.604 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:41 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:41.917 INFO:teuthology.orchestra.run.smithi131.stdout:107374182431 2024-09-11T08:38:41.918 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T08:38:42.566 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:42 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:42.566 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:42 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:42.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:42 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:42.953 INFO:teuthology.orchestra.run.smithi131.stdout:77309411365 2024-09-11T08:38:42.954 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T08:38:43.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:43 smithi131 ceph-mon[37208]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:38:43.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:43 smithi131 ceph-mon[29446]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:38:43.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:43 smithi189 ceph-mon[32675]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:38:44.687 INFO:teuthology.orchestra.run.smithi131.stdout:180388626451 2024-09-11T08:38:44.687 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T08:38:45.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:45 smithi131 ceph-mon[37208]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:38:45.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:45 smithi131 ceph-mon[29446]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:38:45.542 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:45 smithi189 ceph-mon[32675]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:38:47.382 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:47 smithi131 ceph-mon[37208]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:38:47.383 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:47 smithi131 ceph-mon[29446]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:38:47.422 INFO:teuthology.orchestra.run.smithi131.stdout:128849018909 2024-09-11T08:38:47.422 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T08:38:47.554 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:47.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:47 smithi189 ceph-mon[32675]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:38:49.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:49 smithi131 ceph-mon[29446]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:38:49.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:49 smithi131 ceph-mon[37208]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:38:49.524 INFO:teuthology.orchestra.run.smithi131.stdout:154618822681 2024-09-11T08:38:49.524 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T08:38:49.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:49 smithi189 ceph-mon[32675]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:38:49.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:49 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:38:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:38:51.455 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:51 smithi131 ceph-mon[37208]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:38:51.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:51 smithi131 ceph-mon[29446]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:38:51.489 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:51.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:51 smithi189 ceph-mon[32675]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:38:51.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:38:51 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:38:52.380 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:52 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:52.381 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:52 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:52.407 INFO:teuthology.orchestra.run.smithi131.stdout:55834574892 2024-09-11T08:38:52.408 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T08:38:52.437 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:52.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:52 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:38:52.907 INFO:teuthology.orchestra.run.smithi131.stdout:206158430224 2024-09-11T08:38:52.907 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T08:38:53.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:53 smithi131 ceph-mon[29446]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:38:53.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:53 smithi131 ceph-mon[37208]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:38:53.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:53 smithi189 ceph-mon[32675]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:38:54.235 INFO:teuthology.orchestra.run.smithi131.stdout:107374182434 2024-09-11T08:38:54.490 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:54 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3977711490' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:38:54.490 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:54 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3977711490' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:38:54.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:54 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3977711490' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:38:54.806 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182431 got 107374182434 for osd.3 2024-09-11T08:38:54.807 DEBUG:teuthology.parallel:result is None 2024-09-11T08:38:55.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:55 smithi131 ceph-mon[29446]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:38:55.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:55 smithi131 ceph-mon[37208]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:38:55.671 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:55.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:55 smithi189 ceph-mon[32675]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:38:56.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:56 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:56.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:56 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:56.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:56 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:38:56.841 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:57.483 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:57.484 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:57.591 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:57 smithi131 ceph-mon[37208]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:38:57.592 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:57 smithi131 ceph-mon[29446]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:38:57.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:57 smithi189 ceph-mon[32675]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:38:59.540 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:38:59 smithi131 ceph-mon[29446]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:38:59.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:38:59 smithi131 ceph-mon[37208]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:38:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:38:59.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:38:59 smithi189 ceph-mon[32675]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:38:59.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:38:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:38:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:39:01.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:01 smithi189 ceph-mon[32675]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:01.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:39:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:39:01.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:01 smithi131 ceph-mon[37208]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:01.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:01 smithi131 ceph-mon[29446]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:02.331 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:02 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:02.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:02 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:02.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:02 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:02.802 INFO:teuthology.orchestra.run.smithi131.stdout:34359738421 2024-09-11T08:39:03.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:03 smithi131 ceph-mon[29446]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:39:03.489 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:03 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3088121005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:39:03.489 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:03 smithi131 ceph-mon[37208]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:39:03.489 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:03 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3088121005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:39:03.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:03 smithi189 ceph-mon[32675]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:39:03.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:03 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3088121005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:39:05.162 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738415 got 34359738421 for osd.0 2024-09-11T08:39:05.163 DEBUG:teuthology.parallel:result is None 2024-09-11T08:39:05.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:05 smithi131 ceph-mon[37208]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:05.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:05 smithi131 ceph-mon[29446]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:05.501 INFO:teuthology.orchestra.run.smithi131.stdout:154618822684 2024-09-11T08:39:05.736 INFO:teuthology.orchestra.run.smithi131.stdout:128849018913 2024-09-11T08:39:05.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:05 smithi189 ceph-mon[32675]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:06.398 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822681 got 154618822684 for osd.5 2024-09-11T08:39:06.399 DEBUG:teuthology.parallel:result is None 2024-09-11T08:39:06.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:06 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/4125505950' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:39:06.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:06 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1666914885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:39:06.542 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:06 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/4125505950' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:39:06.542 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:06 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1666914885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:39:06.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:06 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/4125505950' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:39:06.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:06 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1666914885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:39:06.856 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018909 got 128849018913 for osd.4 2024-09-11T08:39:06.856 DEBUG:teuthology.parallel:result is None 2024-09-11T08:39:07.617 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:07 smithi131 ceph-mon[37208]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:07.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:07 smithi131 ceph-mon[29446]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:07.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:07 smithi189 ceph-mon[32675]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:08.612 INFO:teuthology.orchestra.run.smithi131.stdout:55834574896 2024-09-11T08:39:08.805 INFO:teuthology.orchestra.run.smithi131.stdout:180388626456 2024-09-11T08:39:09.369 INFO:teuthology.orchestra.run.smithi131.stdout:77309411371 2024-09-11T08:39:09.382 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:09 smithi131 ceph-mon[37208]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:09.382 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:09 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2042160599' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:39:09.382 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:09 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2081637906' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:39:09.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:09 smithi131 ceph-mon[29446]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:09.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:09 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2042160599' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:39:09.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:09 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2081637906' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:39:09.548 INFO:teuthology.orchestra.run.smithi131.stdout:206158430228 2024-09-11T08:39:09.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:09 smithi189 ceph-mon[32675]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:09.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:09 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2042160599' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:39:09.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:09 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2081637906' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:39:09.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:39:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:39:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:39:10.214 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430224 got 206158430228 for osd.7 2024-09-11T08:39:10.214 DEBUG:teuthology.parallel:result is None 2024-09-11T08:39:10.561 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:10 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/717750613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:39:10.561 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:10 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/872940736' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:39:10.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:10 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/717750613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:39:10.562 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:10 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/872940736' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:39:10.562 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:10 smithi131 podman[69941]: 2024-09-11 08:39:10.374220242 +0000 UTC m=+2.734858928 container create c239b7ed231e2cebff6db4aece7b20045a6b3e1ae0ba153d8631c5d58637e9f1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:10.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:10 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/717750613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:39:10.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:10 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/872940736' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:39:10.891 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411365 got 77309411371 for osd.2 2024-09-11T08:39:10.892 DEBUG:teuthology.parallel:result is None 2024-09-11T08:39:11.333 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:39:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:39:11.335 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626451 got 180388626456 for osd.6 2024-09-11T08:39:11.335 DEBUG:teuthology.parallel:result is None 2024-09-11T08:39:11.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:11 smithi131 ceph-mon[37208]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:11.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:11 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:11.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:11 smithi131 ceph-mon[29446]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:11.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:11 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:11.695 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574892 got 55834574896 for osd.1 2024-09-11T08:39:11.695 DEBUG:teuthology.parallel:result is None 2024-09-11T08:39:11.696 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T08:39:11.696 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:39:11.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:11 smithi189 ceph-mon[32675]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:11.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:11 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:12.070 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:12.194 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:11 smithi131 podman[69941]: 2024-09-11 08:39:11.920001627 +0000 UTC m=+4.280640274 container init c239b7ed231e2cebff6db4aece7b20045a6b3e1ae0ba153d8631c5d58637e9f1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:12.194 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:11 smithi131 podman[69941]: 2024-09-11 08:39:11.92399288 +0000 UTC m=+4.284631533 container start c239b7ed231e2cebff6db4aece7b20045a6b3e1ae0ba153d8631c5d58637e9f1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:12.194 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[70160]: ts=2024-09-11T08:39:11.937Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-11T08:39:12.194 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[70160]: ts=2024-09-11T08:39:11.938Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-11T08:39:12.194 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[70160]: ts=2024-09-11T08:39:11.938Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.131 port=9094 2024-09-11T08:39:12.194 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[70160]: ts=2024-09-11T08:39:11.939Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-11T08:39:12.194 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[70160]: ts=2024-09-11T08:39:11.963Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:39:12.195 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[70160]: ts=2024-09-11T08:39:11.964Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:39:12.195 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[70160]: ts=2024-09-11T08:39:11.966Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-11T08:39:12.195 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[70160]: ts=2024-09-11T08:39:11.966Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-11T08:39:12.195 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:12 smithi131 bash[69941]: c239b7ed231e2cebff6db4aece7b20045a6b3e1ae0ba153d8631c5d58637e9f1 2024-09-11T08:39:12.195 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:12 smithi131 systemd[1]: Started Ceph alertmanager.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:39:12.458 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:12] ENGINE Bus STOPPING 2024-09-11T08:39:12.458 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:12.458 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.459 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.459 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.459 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.459 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.459 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.459 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:39:12.459 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.459 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:12.459 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.459 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.459 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.459 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:39:12.460 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:12 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:12.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:12 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:12 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:12 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:12 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:12 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:12 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:12 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:39:12.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:12 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:12.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:12] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:39:12.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:12] ENGINE Bus STOPPED 2024-09-11T08:39:12.891 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:12] ENGINE Bus STARTING 2024-09-11T08:39:12.891 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:12] ENGINE Serving on http://:::9283 2024-09-11T08:39:12.891 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:12] ENGINE Bus STARTED 2024-09-11T08:39:13.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:13 smithi131 ceph-mon[29446]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:13.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:13 smithi131 ceph-mon[29446]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:39:13.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:13 smithi131 ceph-mon[29446]: Deploying daemon grafana.a on smithi189 2024-09-11T08:39:13.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:13 smithi131 ceph-mon[37208]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:13.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:13 smithi131 ceph-mon[37208]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:39:13.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:13 smithi131 ceph-mon[37208]: Deploying daemon grafana.a on smithi189 2024-09-11T08:39:13.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:13 smithi189 ceph-mon[32675]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:13.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:13 smithi189 ceph-mon[32675]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:39:13.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:13 smithi189 ceph-mon[32675]: Deploying daemon grafana.a on smithi189 2024-09-11T08:39:14.038 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:39:14.040 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-11T08:39:14.318 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:13 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[70160]: ts=2024-09-11T08:39:13.940Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000401563s 2024-09-11T08:39:14.762 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-11T08:39:12.214276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":928,"num_read_kb":785,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385372,"kb_used_data":6532,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575844,"statfs":{"total":767960285184,"available":765517664256,"internally_reserved":0,"allocated":6688768,"data_stored":3630562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002579"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349063+0000","last_change":"2024-09-11T08:37:55.434898+0000","last_active":"2024-09-11T08:37:58.349063+0000","last_peered":"2024-09-11T08:37:58.349063+0000","last_clean":"2024-09-11T08:37:58.349063+0000","last_became_active":"2024-09-11T08:37:55.434521+0000","last_became_peered":"2024-09-11T08:37:55.434521+0000","last_unstale":"2024-09-11T08:37:58.349063+0000","last_undegraded":"2024-09-11T08:37:58.349063+0000","last_fullsized":"2024-09-11T08:37:58.349063+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:48:40.076543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.063071+0000","last_change":"2024-09-11T08:37:49.019977+0000","last_active":"2024-09-11T08:37:59.063071+0000","last_peered":"2024-09-11T08:37:59.063071+0000","last_clean":"2024-09-11T08:37:59.063071+0000","last_became_active":"2024-09-11T08:37:49.019077+0000","last_became_peered":"2024-09-11T08:37:49.019077+0000","last_unstale":"2024-09-11T08:37:59.063071+0000","last_undegraded":"2024-09-11T08:37:59.063071+0000","last_fullsized":"2024-09-11T08:37:59.063071+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:11.665331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350332+0000","last_change":"2024-09-11T08:37:51.033534+0000","last_active":"2024-09-11T08:37:58.350332+0000","last_peered":"2024-09-11T08:37:58.350332+0000","last_clean":"2024-09-11T08:37:58.350332+0000","last_became_active":"2024-09-11T08:37:51.033263+0000","last_became_peered":"2024-09-11T08:37:51.033263+0000","last_unstale":"2024-09-11T08:37:58.350332+0000","last_undegraded":"2024-09-11T08:37:58.350332+0000","last_fullsized":"2024-09-11T08:37:58.350332+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:14:10.665290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349300+0000","last_change":"2024-09-11T08:37:53.048567+0000","last_active":"2024-09-11T08:37:58.349300+0000","last_peered":"2024-09-11T08:37:58.349300+0000","last_clean":"2024-09-11T08:37:58.349300+0000","last_became_active":"2024-09-11T08:37:53.048255+0000","last_became_peered":"2024-09-11T08:37:53.048255+0000","last_unstale":"2024-09-11T08:37:58.349300+0000","last_undegraded":"2024-09-11T08:37:58.349300+0000","last_fullsized":"2024-09-11T08:37:58.349300+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:27:48.550065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350393+0000","last_change":"2024-09-11T08:37:49.024131+0000","last_active":"2024-09-11T08:37:58.350393+0000","last_peered":"2024-09-11T08:37:58.350393+0000","last_clean":"2024-09-11T08:37:58.350393+0000","last_became_active":"2024-09-11T08:37:49.023836+0000","last_became_peered":"2024-09-11T08:37:49.023836+0000","last_unstale":"2024-09-11T08:37:58.350393+0000","last_undegraded":"2024-09-11T08:37:58.350393+0000","last_fullsized":"2024-09-11T08:37:58.350393+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:52:50.743128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349300+0000","last_change":"2024-09-11T08:37:51.033574+0000","last_active":"2024-09-11T08:37:58.349300+0000","last_peered":"2024-09-11T08:37:58.349300+0000","last_clean":"2024-09-11T08:37:58.349300+0000","last_became_active":"2024-09-11T08:37:51.033235+0000","last_became_peered":"2024-09-11T08:37:51.033235+0000","last_unstale":"2024-09-11T08:37:58.349300+0000","last_undegraded":"2024-09-11T08:37:58.349300+0000","last_fullsized":"2024-09-11T08:37:58.349300+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:33.844308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.454758+0000","last_change":"2024-09-11T08:37:53.052412+0000","last_active":"2024-09-11T08:37:58.454758+0000","last_peered":"2024-09-11T08:37:58.454758+0000","last_clean":"2024-09-11T08:37:58.454758+0000","last_became_active":"2024-09-11T08:37:53.051983+0000","last_became_peered":"2024-09-11T08:37:53.051983+0000","last_unstale":"2024-09-11T08:37:58.454758+0000","last_undegraded":"2024-09-11T08:37:58.454758+0000","last_fullsized":"2024-09-11T08:37:58.454758+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:10:20.565175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349417+0000","last_change":"2024-09-11T08:37:55.048153+0000","last_active":"2024-09-11T08:37:58.349417+0000","last_peered":"2024-09-11T08:37:58.349417+0000","last_clean":"2024-09-11T08:37:58.349417+0000","last_became_active":"2024-09-11T08:37:55.047646+0000","last_became_peered":"2024-09-11T08:37:55.047646+0000","last_unstale":"2024-09-11T08:37:58.349417+0000","last_undegraded":"2024-09-11T08:37:58.349417+0000","last_fullsized":"2024-09-11T08:37:58.349417+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:40.841075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523541+0000","last_change":"2024-09-11T08:37:49.024673+0000","last_active":"2024-09-11T08:37:59.523541+0000","last_peered":"2024-09-11T08:37:59.523541+0000","last_clean":"2024-09-11T08:37:59.523541+0000","last_became_active":"2024-09-11T08:37:49.024253+0000","last_became_peered":"2024-09-11T08:37:49.024253+0000","last_unstale":"2024-09-11T08:37:59.523541+0000","last_undegraded":"2024-09-11T08:37:59.523541+0000","last_fullsized":"2024-09-11T08:37:59.523541+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:33.729941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"59'5","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349521+0000","last_change":"2024-09-11T08:37:51.030719+0000","last_active":"2024-09-11T08:37:58.349521+0000","last_peered":"2024-09-11T08:37:58.349521+0000","last_clean":"2024-09-11T08:37:58.349521+0000","last_became_active":"2024-09-11T08:37:51.030370+0000","last_became_peered":"2024-09-11T08:37:51.030370+0000","last_unstale":"2024-09-11T08:37:58.349521+0000","last_undegraded":"2024-09-11T08:37:58.349521+0000","last_fullsized":"2024-09-11T08:37:58.349521+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:13:10.948458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523110+0000","last_change":"2024-09-11T08:37:53.045348+0000","last_active":"2024-09-11T08:37:59.523110+0000","last_peered":"2024-09-11T08:37:59.523110+0000","last_clean":"2024-09-11T08:37:59.523110+0000","last_became_active":"2024-09-11T08:37:53.044964+0000","last_became_peered":"2024-09-11T08:37:53.044964+0000","last_unstale":"2024-09-11T08:37:59.523110+0000","last_undegraded":"2024-09-11T08:37:59.523110+0000","last_fullsized":"2024-09-11T08:37:59.523110+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:22.901601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538636+0000","last_change":"2024-09-11T08:37:55.058413+0000","last_active":"2024-09-11T08:37:58.538636+0000","last_peered":"2024-09-11T08:37:58.538636+0000","last_clean":"2024-09-11T08:37:58.538636+0000","last_became_active":"2024-09-11T08:37:55.058030+0000","last_became_peered":"2024-09-11T08:37:55.058030+0000","last_unstale":"2024-09-11T08:37:58.538636+0000","last_undegraded":"2024-09-11T08:37:58.538636+0000","last_fullsized":"2024-09-11T08:37:58.538636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:12.469407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523468+0000","last_change":"2024-09-11T08:37:49.021837+0000","last_active":"2024-09-11T08:37:59.523468+0000","last_peered":"2024-09-11T08:37:59.523468+0000","last_clean":"2024-09-11T08:37:59.523468+0000","last_became_active":"2024-09-11T08:37:49.021579+0000","last_became_peered":"2024-09-11T08:37:49.021579+0000","last_unstale":"2024-09-11T08:37:59.523468+0000","last_undegraded":"2024-09-11T08:37:59.523468+0000","last_fullsized":"2024-09-11T08:37:59.523468+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:12:14.919864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349559+0000","last_change":"2024-09-11T08:37:51.032386+0000","last_active":"2024-09-11T08:37:58.349559+0000","last_peered":"2024-09-11T08:37:58.349559+0000","last_clean":"2024-09-11T08:37:58.349559+0000","last_became_active":"2024-09-11T08:37:51.032155+0000","last_became_peered":"2024-09-11T08:37:51.032155+0000","last_unstale":"2024-09-11T08:37:58.349559+0000","last_undegraded":"2024-09-11T08:37:58.349559+0000","last_fullsized":"2024-09-11T08:37:58.349559+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:51:52.890198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538795+0000","last_change":"2024-09-11T08:37:53.053926+0000","last_active":"2024-09-11T08:37:58.538795+0000","last_peered":"2024-09-11T08:37:58.538795+0000","last_clean":"2024-09-11T08:37:58.538795+0000","last_became_active":"2024-09-11T08:37:53.053645+0000","last_became_peered":"2024-09-11T08:37:53.053645+0000","last_unstale":"2024-09-11T08:37:58.538795+0000","last_undegraded":"2024-09-11T08:37:58.538795+0000","last_fullsized":"2024-09-11T08:37:58.538795+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:14:04.984953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062355+0000","last_change":"2024-09-11T08:37:55.054741+0000","last_active":"2024-09-11T08:37:59.062355+0000","last_peered":"2024-09-11T08:37:59.062355+0000","last_clean":"2024-09-11T08:37:59.062355+0000","last_became_active":"2024-09-11T08:37:55.054441+0000","last_became_peered":"2024-09-11T08:37:55.054441+0000","last_unstale":"2024-09-11T08:37:59.062355+0000","last_undegraded":"2024-09-11T08:37:59.062355+0000","last_fullsized":"2024-09-11T08:37:59.062355+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:04:13.643930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"59'17","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350004+0000","last_change":"2024-09-11T08:37:51.033463+0000","last_active":"2024-09-11T08:37:58.350004+0000","last_peered":"2024-09-11T08:37:58.350004+0000","last_clean":"2024-09-11T08:37:58.350004+0000","last_became_active":"2024-09-11T08:37:51.033153+0000","last_became_peered":"2024-09-11T08:37:51.033153+0000","last_unstale":"2024-09-11T08:37:58.350004+0000","last_undegraded":"2024-09-11T08:37:58.350004+0000","last_fullsized":"2024-09-11T08:37:58.350004+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:19:01.876473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523338+0000","last_change":"2024-09-11T08:37:49.022131+0000","last_active":"2024-09-11T08:37:59.523338+0000","last_peered":"2024-09-11T08:37:59.523338+0000","last_clean":"2024-09-11T08:37:59.523338+0000","last_became_active":"2024-09-11T08:37:49.021654+0000","last_became_peered":"2024-09-11T08:37:49.021654+0000","last_unstale":"2024-09-11T08:37:59.523338+0000","last_undegraded":"2024-09-11T08:37:59.523338+0000","last_fullsized":"2024-09-11T08:37:59.523338+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:33.224730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455373+0000","last_change":"2024-09-11T08:37:53.052918+0000","last_active":"2024-09-11T08:37:58.455373+0000","last_peered":"2024-09-11T08:37:58.455373+0000","last_clean":"2024-09-11T08:37:58.455373+0000","last_became_active":"2024-09-11T08:37:53.052603+0000","last_became_peered":"2024-09-11T08:37:53.052603+0000","last_unstale":"2024-09-11T08:37:58.455373+0000","last_undegraded":"2024-09-11T08:37:58.455373+0000","last_fullsized":"2024-09-11T08:37:58.455373+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:32:30.639347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349540+0000","last_change":"2024-09-11T08:37:55.058670+0000","last_active":"2024-09-11T08:37:58.349540+0000","last_peered":"2024-09-11T08:37:58.349540+0000","last_clean":"2024-09-11T08:37:58.349540+0000","last_became_active":"2024-09-11T08:37:55.058385+0000","last_became_peered":"2024-09-11T08:37:55.058385+0000","last_unstale":"2024-09-11T08:37:58.349540+0000","last_undegraded":"2024-09-11T08:37:58.349540+0000","last_fullsized":"2024-09-11T08:37:58.349540+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:52:46.878375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349729+0000","last_change":"2024-09-11T08:37:51.030783+0000","last_active":"2024-09-11T08:37:58.349729+0000","last_peered":"2024-09-11T08:37:58.349729+0000","last_clean":"2024-09-11T08:37:58.349729+0000","last_became_active":"2024-09-11T08:37:51.030507+0000","last_became_peered":"2024-09-11T08:37:51.030507+0000","last_unstale":"2024-09-11T08:37:58.349729+0000","last_undegraded":"2024-09-11T08:37:58.349729+0000","last_fullsized":"2024-09-11T08:37:58.349729+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:31.188722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"52'1","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455470+0000","last_change":"2024-09-11T08:37:49.021595+0000","last_active":"2024-09-11T08:37:58.455470+0000","last_peered":"2024-09-11T08:37:58.455470+0000","last_clean":"2024-09-11T08:37:58.455470+0000","last_became_active":"2024-09-11T08:37:49.021339+0000","last_became_peered":"2024-09-11T08:37:49.021339+0000","last_unstale":"2024-09-11T08:37:58.455470+0000","last_undegraded":"2024-09-11T08:37:58.455470+0000","last_fullsized":"2024-09-11T08:37:58.455470+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:11:06.396322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183424+0000","last_change":"2024-09-11T08:37:53.054096+0000","last_active":"2024-09-11T08:39:07.183424+0000","last_peered":"2024-09-11T08:39:07.183424+0000","last_clean":"2024-09-11T08:39:07.183424+0000","last_became_active":"2024-09-11T08:37:53.053782+0000","last_became_peered":"2024-09-11T08:37:53.053782+0000","last_unstale":"2024-09-11T08:39:07.183424+0000","last_undegraded":"2024-09-11T08:39:07.183424+0000","last_fullsized":"2024-09-11T08:39:07.183424+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:57:54.554544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349748+0000","last_change":"2024-09-11T08:37:55.048149+0000","last_active":"2024-09-11T08:37:58.349748+0000","last_peered":"2024-09-11T08:37:58.349748+0000","last_clean":"2024-09-11T08:37:58.349748+0000","last_became_active":"2024-09-11T08:37:55.047884+0000","last_became_peered":"2024-09-11T08:37:55.047884+0000","last_unstale":"2024-09-11T08:37:58.349748+0000","last_undegraded":"2024-09-11T08:37:58.349748+0000","last_fullsized":"2024-09-11T08:37:58.349748+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:47:34.276764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455866+0000","last_change":"2024-09-11T08:37:51.032131+0000","last_active":"2024-09-11T08:37:58.455866+0000","last_peered":"2024-09-11T08:37:58.455866+0000","last_clean":"2024-09-11T08:37:58.455866+0000","last_became_active":"2024-09-11T08:37:51.031859+0000","last_became_peered":"2024-09-11T08:37:51.031859+0000","last_unstale":"2024-09-11T08:37:58.455866+0000","last_undegraded":"2024-09-11T08:37:58.455866+0000","last_fullsized":"2024-09-11T08:37:58.455866+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:36:12.907878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.454941+0000","last_change":"2024-09-11T08:37:49.019359+0000","last_active":"2024-09-11T08:37:58.454941+0000","last_peered":"2024-09-11T08:37:58.454941+0000","last_clean":"2024-09-11T08:37:58.454941+0000","last_became_active":"2024-09-11T08:37:49.018945+0000","last_became_peered":"2024-09-11T08:37:49.018945+0000","last_unstale":"2024-09-11T08:37:58.454941+0000","last_undegraded":"2024-09-11T08:37:58.454941+0000","last_fullsized":"2024-09-11T08:37:58.454941+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:16:48.101619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183569+0000","last_change":"2024-09-11T08:37:53.048189+0000","last_active":"2024-09-11T08:39:07.183569+0000","last_peered":"2024-09-11T08:39:07.183569+0000","last_clean":"2024-09-11T08:39:07.183569+0000","last_became_active":"2024-09-11T08:37:53.047910+0000","last_became_peered":"2024-09-11T08:37:53.047910+0000","last_unstale":"2024-09-11T08:39:07.183569+0000","last_undegraded":"2024-09-11T08:39:07.183569+0000","last_fullsized":"2024-09-11T08:39:07.183569+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:47.431633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538934+0000","last_change":"2024-09-11T08:37:55.054848+0000","last_active":"2024-09-11T08:37:58.538934+0000","last_peered":"2024-09-11T08:37:58.538934+0000","last_clean":"2024-09-11T08:37:58.538934+0000","last_became_active":"2024-09-11T08:37:55.054584+0000","last_became_peered":"2024-09-11T08:37:55.054584+0000","last_unstale":"2024-09-11T08:37:58.538934+0000","last_undegraded":"2024-09-11T08:37:58.538934+0000","last_fullsized":"2024-09-11T08:37:58.538934+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:45:09.313154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349820+0000","last_change":"2024-09-11T08:37:51.033616+0000","last_active":"2024-09-11T08:37:58.349820+0000","last_peered":"2024-09-11T08:37:58.349820+0000","last_clean":"2024-09-11T08:37:58.349820+0000","last_became_active":"2024-09-11T08:37:51.033269+0000","last_became_peered":"2024-09-11T08:37:51.033269+0000","last_unstale":"2024-09-11T08:37:58.349820+0000","last_undegraded":"2024-09-11T08:37:58.349820+0000","last_fullsized":"2024-09-11T08:37:58.349820+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:40.380938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.522991+0000","last_change":"2024-09-11T08:37:49.020669+0000","last_active":"2024-09-11T08:37:59.522991+0000","last_peered":"2024-09-11T08:37:59.522991+0000","last_clean":"2024-09-11T08:37:59.522991+0000","last_became_active":"2024-09-11T08:37:49.020094+0000","last_became_peered":"2024-09-11T08:37:49.020094+0000","last_unstale":"2024-09-11T08:37:59.522991+0000","last_undegraded":"2024-09-11T08:37:59.522991+0000","last_fullsized":"2024-09-11T08:37:59.522991+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:08:46.495559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.539165+0000","last_change":"2024-09-11T08:37:53.052207+0000","last_active":"2024-09-11T08:37:58.539165+0000","last_peered":"2024-09-11T08:37:58.539165+0000","last_clean":"2024-09-11T08:37:58.539165+0000","last_became_active":"2024-09-11T08:37:53.051934+0000","last_became_peered":"2024-09-11T08:37:53.051934+0000","last_unstale":"2024-09-11T08:37:58.539165+0000","last_undegraded":"2024-09-11T08:37:58.539165+0000","last_fullsized":"2024-09-11T08:37:58.539165+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:31:12.635418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350830+0000","last_change":"2024-09-11T08:37:55.436076+0000","last_active":"2024-09-11T08:37:58.350830+0000","last_peered":"2024-09-11T08:37:58.350830+0000","last_clean":"2024-09-11T08:37:58.350830+0000","last_became_active":"2024-09-11T08:37:55.435653+0000","last_became_peered":"2024-09-11T08:37:55.435653+0000","last_unstale":"2024-09-11T08:37:58.350830+0000","last_undegraded":"2024-09-11T08:37:58.350830+0000","last_fullsized":"2024-09-11T08:37:58.350830+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:21:15.501305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"59'14","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349384+0000","last_change":"2024-09-11T08:37:51.029239+0000","last_active":"2024-09-11T08:37:58.349384+0000","last_peered":"2024-09-11T08:37:58.349384+0000","last_clean":"2024-09-11T08:37:58.349384+0000","last_became_active":"2024-09-11T08:37:51.028910+0000","last_became_peered":"2024-09-11T08:37:51.028910+0000","last_unstale":"2024-09-11T08:37:58.349384+0000","last_undegraded":"2024-09-11T08:37:58.349384+0000","last_fullsized":"2024-09-11T08:37:58.349384+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:47.188353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.482338+0000","last_change":"2024-09-11T08:37:49.019139+0000","last_active":"2024-09-11T08:37:58.482338+0000","last_peered":"2024-09-11T08:37:58.482338+0000","last_clean":"2024-09-11T08:37:58.482338+0000","last_became_active":"2024-09-11T08:37:49.018914+0000","last_became_peered":"2024-09-11T08:37:49.018914+0000","last_unstale":"2024-09-11T08:37:58.482338+0000","last_undegraded":"2024-09-11T08:37:58.482338+0000","last_fullsized":"2024-09-11T08:37:58.482338+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:29:13.244505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"59'8","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183481+0000","last_change":"2024-09-11T08:37:53.047443+0000","last_active":"2024-09-11T08:39:07.183481+0000","last_peered":"2024-09-11T08:39:07.183481+0000","last_clean":"2024-09-11T08:39:07.183481+0000","last_became_active":"2024-09-11T08:37:53.047124+0000","last_became_peered":"2024-09-11T08:37:53.047124+0000","last_unstale":"2024-09-11T08:39:07.183481+0000","last_undegraded":"2024-09-11T08:39:07.183481+0000","last_fullsized":"2024-09-11T08:39:07.183481+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:43.958314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523197+0000","last_change":"2024-09-11T08:37:55.433815+0000","last_active":"2024-09-11T08:37:59.523197+0000","last_peered":"2024-09-11T08:37:59.523197+0000","last_clean":"2024-09-11T08:37:59.523197+0000","last_became_active":"2024-09-11T08:37:55.433529+0000","last_became_peered":"2024-09-11T08:37:55.433529+0000","last_unstale":"2024-09-11T08:37:59.523197+0000","last_undegraded":"2024-09-11T08:37:59.523197+0000","last_fullsized":"2024-09-11T08:37:59.523197+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:23:54.418499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"59'18","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349109+0000","last_change":"2024-09-11T08:37:51.271316+0000","last_active":"2024-09-11T08:37:58.349109+0000","last_peered":"2024-09-11T08:37:58.349109+0000","last_clean":"2024-09-11T08:37:58.349109+0000","last_became_active":"2024-09-11T08:37:51.271057+0000","last_became_peered":"2024-09-11T08:37:51.271057+0000","last_unstale":"2024-09-11T08:37:58.349109+0000","last_undegraded":"2024-09-11T08:37:58.349109+0000","last_fullsized":"2024-09-11T08:37:58.349109+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:30:24.674369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455544+0000","last_change":"2024-09-11T08:37:49.014008+0000","last_active":"2024-09-11T08:37:58.455544+0000","last_peered":"2024-09-11T08:37:58.455544+0000","last_clean":"2024-09-11T08:37:58.455544+0000","last_became_active":"2024-09-11T08:37:49.013731+0000","last_became_peered":"2024-09-11T08:37:49.013731+0000","last_unstale":"2024-09-11T08:37:58.455544+0000","last_undegraded":"2024-09-11T08:37:58.455544+0000","last_fullsized":"2024-09-11T08:37:58.455544+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:28:20.967515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349316+0000","last_change":"2024-09-11T08:37:53.047434+0000","last_active":"2024-09-11T08:37:58.349316+0000","last_peered":"2024-09-11T08:37:58.349316+0000","last_clean":"2024-09-11T08:37:58.349316+0000","last_became_active":"2024-09-11T08:37:53.047158+0000","last_became_peered":"2024-09-11T08:37:53.047158+0000","last_unstale":"2024-09-11T08:37:58.349316+0000","last_undegraded":"2024-09-11T08:37:58.349316+0000","last_fullsized":"2024-09-11T08:37:58.349316+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:30:36.115849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349524+0000","last_change":"2024-09-11T08:37:55.054722+0000","last_active":"2024-09-11T08:37:58.349524+0000","last_peered":"2024-09-11T08:37:58.349524+0000","last_clean":"2024-09-11T08:37:58.349524+0000","last_became_active":"2024-09-11T08:37:55.054470+0000","last_became_peered":"2024-09-11T08:37:55.054470+0000","last_unstale":"2024-09-11T08:37:58.349524+0000","last_undegraded":"2024-09-11T08:37:58.349524+0000","last_fullsized":"2024-09-11T08:37:58.349524+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:54:52.735680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"59'19","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062777+0000","last_change":"2024-09-11T08:37:51.271190+0000","last_active":"2024-09-11T08:37:59.062777+0000","last_peered":"2024-09-11T08:37:59.062777+0000","last_clean":"2024-09-11T08:37:59.062777+0000","last_became_active":"2024-09-11T08:37:51.270341+0000","last_became_peered":"2024-09-11T08:37:51.270341+0000","last_unstale":"2024-09-11T08:37:59.062777+0000","last_undegraded":"2024-09-11T08:37:59.062777+0000","last_fullsized":"2024-09-11T08:37:59.062777+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:43.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523593+0000","last_change":"2024-09-11T08:37:49.023943+0000","last_active":"2024-09-11T08:37:59.523593+0000","last_peered":"2024-09-11T08:37:59.523593+0000","last_clean":"2024-09-11T08:37:59.523593+0000","last_became_active":"2024-09-11T08:37:49.022862+0000","last_became_peered":"2024-09-11T08:37:49.022862+0000","last_unstale":"2024-09-11T08:37:59.523593+0000","last_undegraded":"2024-09-11T08:37:59.523593+0000","last_fullsized":"2024-09-11T08:37:59.523593+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:44:14.646330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.481869+0000","last_change":"2024-09-11T08:37:53.053668+0000","last_active":"2024-09-11T08:37:58.481869+0000","last_peered":"2024-09-11T08:37:58.481869+0000","last_clean":"2024-09-11T08:37:58.481869+0000","last_became_active":"2024-09-11T08:37:53.053389+0000","last_became_peered":"2024-09-11T08:37:53.053389+0000","last_unstale":"2024-09-11T08:37:58.481869+0000","last_undegraded":"2024-09-11T08:37:58.481869+0000","last_fullsized":"2024-09-11T08:37:58.481869+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:47:26.099011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.454704+0000","last_change":"2024-09-11T08:37:55.433983+0000","last_active":"2024-09-11T08:37:58.454704+0000","last_peered":"2024-09-11T08:37:58.454704+0000","last_clean":"2024-09-11T08:37:58.454704+0000","last_became_active":"2024-09-11T08:37:55.433578+0000","last_became_peered":"2024-09-11T08:37:55.433578+0000","last_unstale":"2024-09-11T08:37:58.454704+0000","last_undegraded":"2024-09-11T08:37:58.454704+0000","last_fullsized":"2024-09-11T08:37:58.454704+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:05:42.370642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455406+0000","last_change":"2024-09-11T08:37:51.029311+0000","last_active":"2024-09-11T08:37:58.455406+0000","last_peered":"2024-09-11T08:37:58.455406+0000","last_clean":"2024-09-11T08:37:58.455406+0000","last_became_active":"2024-09-11T08:37:51.028561+0000","last_became_peered":"2024-09-11T08:37:51.028561+0000","last_unstale":"2024-09-11T08:37:58.455406+0000","last_undegraded":"2024-09-11T08:37:58.455406+0000","last_fullsized":"2024-09-11T08:37:58.455406+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:30:42.923055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455397+0000","last_change":"2024-09-11T08:37:49.019254+0000","last_active":"2024-09-11T08:37:58.455397+0000","last_peered":"2024-09-11T08:37:58.455397+0000","last_clean":"2024-09-11T08:37:58.455397+0000","last_became_active":"2024-09-11T08:37:49.018991+0000","last_became_peered":"2024-09-11T08:37:49.018991+0000","last_unstale":"2024-09-11T08:37:58.455397+0000","last_undegraded":"2024-09-11T08:37:58.455397+0000","last_fullsized":"2024-09-11T08:37:58.455397+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:27.169174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183431+0000","last_change":"2024-09-11T08:37:53.049824+0000","last_active":"2024-09-11T08:39:07.183431+0000","last_peered":"2024-09-11T08:39:07.183431+0000","last_clean":"2024-09-11T08:39:07.183431+0000","last_became_active":"2024-09-11T08:37:53.049537+0000","last_became_peered":"2024-09-11T08:37:53.049537+0000","last_unstale":"2024-09-11T08:39:07.183431+0000","last_undegraded":"2024-09-11T08:39:07.183431+0000","last_fullsized":"2024-09-11T08:39:07.183431+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:05.414755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062162+0000","last_change":"2024-09-11T08:37:55.057894+0000","last_active":"2024-09-11T08:37:59.062162+0000","last_peered":"2024-09-11T08:37:59.062162+0000","last_clean":"2024-09-11T08:37:59.062162+0000","last_became_active":"2024-09-11T08:37:55.057573+0000","last_became_peered":"2024-09-11T08:37:55.057573+0000","last_unstale":"2024-09-11T08:37:59.062162+0000","last_undegraded":"2024-09-11T08:37:59.062162+0000","last_fullsized":"2024-09-11T08:37:59.062162+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:25.123202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"59'28","reported_seq":81,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183534+0000","last_change":"2024-09-11T08:37:51.031243+0000","last_active":"2024-09-11T08:39:07.183534+0000","last_peered":"2024-09-11T08:39:07.183534+0000","last_clean":"2024-09-11T08:39:07.183534+0000","last_became_active":"2024-09-11T08:37:51.030859+0000","last_became_peered":"2024-09-11T08:37:51.030859+0000","last_unstale":"2024-09-11T08:39:07.183534+0000","last_undegraded":"2024-09-11T08:39:07.183534+0000","last_fullsized":"2024-09-11T08:39:07.183534+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:50:08.622543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538770+0000","last_change":"2024-09-11T08:37:49.020429+0000","last_active":"2024-09-11T08:37:58.538770+0000","last_peered":"2024-09-11T08:37:58.538770+0000","last_clean":"2024-09-11T08:37:58.538770+0000","last_became_active":"2024-09-11T08:37:49.020167+0000","last_became_peered":"2024-09-11T08:37:49.020167+0000","last_unstale":"2024-09-11T08:37:58.538770+0000","last_undegraded":"2024-09-11T08:37:58.538770+0000","last_fullsized":"2024-09-11T08:37:58.538770+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:39.265497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062928+0000","last_change":"2024-09-11T08:37:53.047856+0000","last_active":"2024-09-11T08:37:59.062928+0000","last_peered":"2024-09-11T08:37:59.062928+0000","last_clean":"2024-09-11T08:37:59.062928+0000","last_became_active":"2024-09-11T08:37:53.046447+0000","last_became_peered":"2024-09-11T08:37:53.046447+0000","last_unstale":"2024-09-11T08:37:59.062928+0000","last_undegraded":"2024-09-11T08:37:59.062928+0000","last_fullsized":"2024-09-11T08:37:59.062928+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:32:51.819175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350129+0000","last_change":"2024-09-11T08:37:55.055772+0000","last_active":"2024-09-11T08:37:58.350129+0000","last_peered":"2024-09-11T08:37:58.350129+0000","last_clean":"2024-09-11T08:37:58.350129+0000","last_became_active":"2024-09-11T08:37:55.055503+0000","last_became_peered":"2024-09-11T08:37:55.055503+0000","last_unstale":"2024-09-11T08:37:58.350129+0000","last_undegraded":"2024-09-11T08:37:58.350129+0000","last_fullsized":"2024-09-11T08:37:58.350129+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:05.095131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'13","reported_seq":43,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455916+0000","last_change":"2024-09-11T08:37:51.034069+0000","last_active":"2024-09-11T08:37:58.455916+0000","last_peered":"2024-09-11T08:37:58.455916+0000","last_clean":"2024-09-11T08:37:58.455916+0000","last_became_active":"2024-09-11T08:37:51.033633+0000","last_became_peered":"2024-09-11T08:37:51.033633+0000","last_unstale":"2024-09-11T08:37:58.455916+0000","last_undegraded":"2024-09-11T08:37:58.455916+0000","last_fullsized":"2024-09-11T08:37:58.455916+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:20:42.157777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"60'5","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:11.112089+0000","last_change":"2024-09-11T08:37:53.040217+0000","last_active":"2024-09-11T08:39:11.112089+0000","last_peered":"2024-09-11T08:39:11.112089+0000","last_clean":"2024-09-11T08:39:11.112089+0000","last_became_active":"2024-09-11T08:37:49.016198+0000","last_became_peered":"2024-09-11T08:37:49.016198+0000","last_unstale":"2024-09-11T08:39:11.112089+0000","last_undegraded":"2024-09-11T08:39:11.112089+0000","last_fullsized":"2024-09-11T08:39:11.112089+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:08:19.801628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017901740000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349878+0000","last_change":"2024-09-11T08:37:49.020128+0000","last_active":"2024-09-11T08:37:58.349878+0000","last_peered":"2024-09-11T08:37:58.349878+0000","last_clean":"2024-09-11T08:37:58.349878+0000","last_became_active":"2024-09-11T08:37:49.019785+0000","last_became_peered":"2024-09-11T08:37:49.019785+0000","last_unstale":"2024-09-11T08:37:58.349878+0000","last_undegraded":"2024-09-11T08:37:58.349878+0000","last_fullsized":"2024-09-11T08:37:58.349878+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:37:24.004661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349739+0000","last_change":"2024-09-11T08:37:53.054033+0000","last_active":"2024-09-11T08:37:58.349739+0000","last_peered":"2024-09-11T08:37:58.349739+0000","last_clean":"2024-09-11T08:37:58.349739+0000","last_became_active":"2024-09-11T08:37:53.053718+0000","last_became_peered":"2024-09-11T08:37:53.053718+0000","last_unstale":"2024-09-11T08:37:58.349739+0000","last_undegraded":"2024-09-11T08:37:58.349739+0000","last_fullsized":"2024-09-11T08:37:58.349739+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:00:23.513884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523363+0000","last_change":"2024-09-11T08:37:55.433912+0000","last_active":"2024-09-11T08:37:59.523363+0000","last_peered":"2024-09-11T08:37:59.523363+0000","last_clean":"2024-09-11T08:37:59.523363+0000","last_became_active":"2024-09-11T08:37:55.433510+0000","last_became_peered":"2024-09-11T08:37:55.433510+0000","last_unstale":"2024-09-11T08:37:59.523363+0000","last_undegraded":"2024-09-11T08:37:59.523363+0000","last_fullsized":"2024-09-11T08:37:59.523363+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:28:34.418337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"59'12","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062241+0000","last_change":"2024-09-11T08:37:51.037580+0000","last_active":"2024-09-11T08:37:59.062241+0000","last_peered":"2024-09-11T08:37:59.062241+0000","last_clean":"2024-09-11T08:37:59.062241+0000","last_became_active":"2024-09-11T08:37:51.037288+0000","last_became_peered":"2024-09-11T08:37:51.037288+0000","last_unstale":"2024-09-11T08:37:59.062241+0000","last_undegraded":"2024-09-11T08:37:59.062241+0000","last_fullsized":"2024-09-11T08:37:59.062241+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:48:14.566225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"54'1","reported_seq":24,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062619+0000","last_change":"2024-09-11T08:37:53.039112+0000","last_active":"2024-09-11T08:37:59.062619+0000","last_peered":"2024-09-11T08:37:59.062619+0000","last_clean":"2024-09-11T08:37:59.062619+0000","last_became_active":"2024-09-11T08:37:49.020745+0000","last_became_peered":"2024-09-11T08:37:49.020745+0000","last_unstale":"2024-09-11T08:37:59.062619+0000","last_undegraded":"2024-09-11T08:37:59.062619+0000","last_fullsized":"2024-09-11T08:37:59.062619+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:01.595693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061342700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523051+0000","last_change":"2024-09-11T08:37:49.023769+0000","last_active":"2024-09-11T08:37:59.523051+0000","last_peered":"2024-09-11T08:37:59.523051+0000","last_clean":"2024-09-11T08:37:59.523051+0000","last_became_active":"2024-09-11T08:37:49.022720+0000","last_became_peered":"2024-09-11T08:37:49.022720+0000","last_unstale":"2024-09-11T08:37:59.523051+0000","last_undegraded":"2024-09-11T08:37:59.523051+0000","last_fullsized":"2024-09-11T08:37:59.523051+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:17:32.792408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.539165+0000","last_change":"2024-09-11T08:37:53.054982+0000","last_active":"2024-09-11T08:37:58.539165+0000","last_peered":"2024-09-11T08:37:58.539165+0000","last_clean":"2024-09-11T08:37:58.539165+0000","last_became_active":"2024-09-11T08:37:53.054729+0000","last_became_peered":"2024-09-11T08:37:53.054729+0000","last_unstale":"2024-09-11T08:37:58.539165+0000","last_undegraded":"2024-09-11T08:37:58.539165+0000","last_fullsized":"2024-09-11T08:37:58.539165+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:04:47.381949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455025+0000","last_change":"2024-09-11T08:37:55.056675+0000","last_active":"2024-09-11T08:37:58.455025+0000","last_peered":"2024-09-11T08:37:58.455025+0000","last_clean":"2024-09-11T08:37:58.455025+0000","last_became_active":"2024-09-11T08:37:55.056339+0000","last_became_peered":"2024-09-11T08:37:55.056339+0000","last_unstale":"2024-09-11T08:37:58.455025+0000","last_undegraded":"2024-09-11T08:37:58.455025+0000","last_fullsized":"2024-09-11T08:37:58.455025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:34:22.793615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"59'14","reported_seq":38,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.482069+0000","last_change":"2024-09-11T08:37:51.271898+0000","last_active":"2024-09-11T08:37:58.482069+0000","last_peered":"2024-09-11T08:37:58.482069+0000","last_clean":"2024-09-11T08:37:58.482069+0000","last_became_active":"2024-09-11T08:37:51.271313+0000","last_became_peered":"2024-09-11T08:37:51.271313+0000","last_unstale":"2024-09-11T08:37:58.482069+0000","last_undegraded":"2024-09-11T08:37:58.482069+0000","last_fullsized":"2024-09-11T08:37:58.482069+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:17:03.912213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'2","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349550+0000","last_change":"2024-09-11T08:37:53.044478+0000","last_active":"2024-09-11T08:37:58.349550+0000","last_peered":"2024-09-11T08:37:58.349550+0000","last_clean":"2024-09-11T08:37:58.349550+0000","last_became_active":"2024-09-11T08:37:49.021133+0000","last_became_peered":"2024-09-11T08:37:49.021133+0000","last_unstale":"2024-09-11T08:37:58.349550+0000","last_undegraded":"2024-09-11T08:37:58.349550+0000","last_fullsized":"2024-09-11T08:37:58.349550+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:20:00.991673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026546719999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538944+0000","last_change":"2024-09-11T08:37:49.023418+0000","last_active":"2024-09-11T08:37:58.538944+0000","last_peered":"2024-09-11T08:37:58.538944+0000","last_clean":"2024-09-11T08:37:58.538944+0000","last_became_active":"2024-09-11T08:37:49.023138+0000","last_became_peered":"2024-09-11T08:37:49.023138+0000","last_unstale":"2024-09-11T08:37:58.538944+0000","last_undegraded":"2024-09-11T08:37:58.538944+0000","last_fullsized":"2024-09-11T08:37:58.538944+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:06.531998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"61'79","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:38:28.254556+0000","last_change":"2024-09-11T08:37:39.200133+0000","last_active":"2024-09-11T08:38:28.254556+0000","last_peered":"2024-09-11T08:38:28.254556+0000","last_clean":"2024-09-11T08:38:28.254556+0000","last_became_active":"2024-09-11T08:37:39.193681+0000","last_became_peered":"2024-09-11T08:37:39.193681+0000","last_unstale":"2024-09-11T08:38:28.254556+0000","last_undegraded":"2024-09-11T08:38:28.254556+0000","last_fullsized":"2024-09-11T08:38:28.254556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:35:44.055988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:35:44.055988+0000","last_clean_scrub_stamp":"2024-09-11T08:35:44.055988+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:03:15.848922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523327+0000","last_change":"2024-09-11T08:37:53.050902+0000","last_active":"2024-09-11T08:37:59.523327+0000","last_peered":"2024-09-11T08:37:59.523327+0000","last_clean":"2024-09-11T08:37:59.523327+0000","last_became_active":"2024-09-11T08:37:53.050575+0000","last_became_peered":"2024-09-11T08:37:53.050575+0000","last_unstale":"2024-09-11T08:37:59.523327+0000","last_undegraded":"2024-09-11T08:37:59.523327+0000","last_fullsized":"2024-09-11T08:37:59.523327+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:13:07.312142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.539241+0000","last_change":"2024-09-11T08:37:55.059161+0000","last_active":"2024-09-11T08:37:58.539241+0000","last_peered":"2024-09-11T08:37:58.539241+0000","last_clean":"2024-09-11T08:37:58.539241+0000","last_became_active":"2024-09-11T08:37:55.058853+0000","last_became_peered":"2024-09-11T08:37:55.058853+0000","last_unstale":"2024-09-11T08:37:58.539241+0000","last_undegraded":"2024-09-11T08:37:58.539241+0000","last_fullsized":"2024-09-11T08:37:58.539241+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:44.113253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"59'19","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.482002+0000","last_change":"2024-09-11T08:37:51.271006+0000","last_active":"2024-09-11T08:37:58.482002+0000","last_peered":"2024-09-11T08:37:58.482002+0000","last_clean":"2024-09-11T08:37:58.482002+0000","last_became_active":"2024-09-11T08:37:51.270631+0000","last_became_peered":"2024-09-11T08:37:51.270631+0000","last_unstale":"2024-09-11T08:37:58.482002+0000","last_undegraded":"2024-09-11T08:37:58.482002+0000","last_fullsized":"2024-09-11T08:37:58.482002+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:49:53.798485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350354+0000","last_change":"2024-09-11T08:37:49.018768+0000","last_active":"2024-09-11T08:37:58.350354+0000","last_peered":"2024-09-11T08:37:58.350354+0000","last_clean":"2024-09-11T08:37:58.350354+0000","last_became_active":"2024-09-11T08:37:49.017506+0000","last_became_peered":"2024-09-11T08:37:49.017506+0000","last_unstale":"2024-09-11T08:37:58.350354+0000","last_undegraded":"2024-09-11T08:37:58.350354+0000","last_fullsized":"2024-09-11T08:37:58.350354+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:18.561931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349411+0000","last_change":"2024-09-11T08:37:53.051776+0000","last_active":"2024-09-11T08:37:58.349411+0000","last_peered":"2024-09-11T08:37:58.349411+0000","last_clean":"2024-09-11T08:37:58.349411+0000","last_became_active":"2024-09-11T08:37:53.051476+0000","last_became_peered":"2024-09-11T08:37:53.051476+0000","last_unstale":"2024-09-11T08:37:58.349411+0000","last_undegraded":"2024-09-11T08:37:58.349411+0000","last_fullsized":"2024-09-11T08:37:58.349411+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:21:25.949454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523120+0000","last_change":"2024-09-11T08:37:55.058963+0000","last_active":"2024-09-11T08:37:59.523120+0000","last_peered":"2024-09-11T08:37:59.523120+0000","last_clean":"2024-09-11T08:37:59.523120+0000","last_became_active":"2024-09-11T08:37:55.058576+0000","last_became_peered":"2024-09-11T08:37:55.058576+0000","last_unstale":"2024-09-11T08:37:59.523120+0000","last_undegraded":"2024-09-11T08:37:59.523120+0000","last_fullsized":"2024-09-11T08:37:59.523120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:03:31.703591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.061984+0000","last_change":"2024-09-11T08:37:51.033565+0000","last_active":"2024-09-11T08:37:59.061984+0000","last_peered":"2024-09-11T08:37:59.061984+0000","last_clean":"2024-09-11T08:37:59.061984+0000","last_became_active":"2024-09-11T08:37:51.033264+0000","last_became_peered":"2024-09-11T08:37:51.033264+0000","last_unstale":"2024-09-11T08:37:59.061984+0000","last_undegraded":"2024-09-11T08:37:59.061984+0000","last_fullsized":"2024-09-11T08:37:59.061984+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:53.014927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455776+0000","last_change":"2024-09-11T08:37:49.017286+0000","last_active":"2024-09-11T08:37:58.455776+0000","last_peered":"2024-09-11T08:37:58.455776+0000","last_clean":"2024-09-11T08:37:58.455776+0000","last_became_active":"2024-09-11T08:37:49.017025+0000","last_became_peered":"2024-09-11T08:37:49.017025+0000","last_unstale":"2024-09-11T08:37:58.455776+0000","last_undegraded":"2024-09-11T08:37:58.455776+0000","last_fullsized":"2024-09-11T08:37:58.455776+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:39:26.463336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349344+0000","last_change":"2024-09-11T08:37:53.052060+0000","last_active":"2024-09-11T08:37:58.349344+0000","last_peered":"2024-09-11T08:37:58.349344+0000","last_clean":"2024-09-11T08:37:58.349344+0000","last_became_active":"2024-09-11T08:37:53.051738+0000","last_became_peered":"2024-09-11T08:37:53.051738+0000","last_unstale":"2024-09-11T08:37:58.349344+0000","last_undegraded":"2024-09-11T08:37:58.349344+0000","last_fullsized":"2024-09-11T08:37:58.349344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:02.789274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062493+0000","last_change":"2024-09-11T08:37:55.054762+0000","last_active":"2024-09-11T08:37:59.062493+0000","last_peered":"2024-09-11T08:37:59.062493+0000","last_clean":"2024-09-11T08:37:59.062493+0000","last_became_active":"2024-09-11T08:37:55.054348+0000","last_became_peered":"2024-09-11T08:37:55.054348+0000","last_unstale":"2024-09-11T08:37:59.062493+0000","last_undegraded":"2024-09-11T08:37:59.062493+0000","last_fullsized":"2024-09-11T08:37:59.062493+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:39.743624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538526+0000","last_change":"2024-09-11T08:37:51.271344+0000","last_active":"2024-09-11T08:37:58.538526+0000","last_peered":"2024-09-11T08:37:58.538526+0000","last_clean":"2024-09-11T08:37:58.538526+0000","last_became_active":"2024-09-11T08:37:51.271042+0000","last_became_peered":"2024-09-11T08:37:51.271042+0000","last_unstale":"2024-09-11T08:37:58.538526+0000","last_undegraded":"2024-09-11T08:37:58.538526+0000","last_fullsized":"2024-09-11T08:37:58.538526+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:13.715865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349936+0000","last_change":"2024-09-11T08:37:49.018660+0000","last_active":"2024-09-11T08:37:58.349936+0000","last_peered":"2024-09-11T08:37:58.349936+0000","last_clean":"2024-09-11T08:37:58.349936+0000","last_became_active":"2024-09-11T08:37:49.018423+0000","last_became_peered":"2024-09-11T08:37:49.018423+0000","last_unstale":"2024-09-11T08:37:58.349936+0000","last_undegraded":"2024-09-11T08:37:58.349936+0000","last_fullsized":"2024-09-11T08:37:58.349936+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:02:26.645297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183576+0000","last_change":"2024-09-11T08:37:53.049055+0000","last_active":"2024-09-11T08:39:07.183576+0000","last_peered":"2024-09-11T08:39:07.183576+0000","last_clean":"2024-09-11T08:39:07.183576+0000","last_became_active":"2024-09-11T08:37:53.048765+0000","last_became_peered":"2024-09-11T08:37:53.048765+0000","last_unstale":"2024-09-11T08:39:07.183576+0000","last_undegraded":"2024-09-11T08:39:07.183576+0000","last_fullsized":"2024-09-11T08:39:07.183576+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:56:21.957087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538610+0000","last_change":"2024-09-11T08:37:55.433918+0000","last_active":"2024-09-11T08:37:58.538610+0000","last_peered":"2024-09-11T08:37:58.538610+0000","last_clean":"2024-09-11T08:37:58.538610+0000","last_became_active":"2024-09-11T08:37:55.433544+0000","last_became_peered":"2024-09-11T08:37:55.433544+0000","last_unstale":"2024-09-11T08:37:58.538610+0000","last_undegraded":"2024-09-11T08:37:58.538610+0000","last_fullsized":"2024-09-11T08:37:58.538610+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:44:48.320347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349354+0000","last_change":"2024-09-11T08:37:51.033723+0000","last_active":"2024-09-11T08:37:58.349354+0000","last_peered":"2024-09-11T08:37:58.349354+0000","last_clean":"2024-09-11T08:37:58.349354+0000","last_became_active":"2024-09-11T08:37:51.033358+0000","last_became_peered":"2024-09-11T08:37:51.033358+0000","last_unstale":"2024-09-11T08:37:58.349354+0000","last_undegraded":"2024-09-11T08:37:58.349354+0000","last_fullsized":"2024-09-11T08:37:58.349354+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:57:03.991746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"52'2","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350091+0000","last_change":"2024-09-11T08:37:49.019809+0000","last_active":"2024-09-11T08:37:58.350091+0000","last_peered":"2024-09-11T08:37:58.350091+0000","last_clean":"2024-09-11T08:37:58.350091+0000","last_became_active":"2024-09-11T08:37:49.019553+0000","last_became_peered":"2024-09-11T08:37:49.019553+0000","last_unstale":"2024-09-11T08:37:58.350091+0000","last_undegraded":"2024-09-11T08:37:58.350091+0000","last_fullsized":"2024-09-11T08:37:58.350091+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:12:17.646393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349366+0000","last_change":"2024-09-11T08:37:53.051008+0000","last_active":"2024-09-11T08:37:58.349366+0000","last_peered":"2024-09-11T08:37:58.349366+0000","last_clean":"2024-09-11T08:37:58.349366+0000","last_became_active":"2024-09-11T08:37:53.050733+0000","last_became_peered":"2024-09-11T08:37:53.050733+0000","last_unstale":"2024-09-11T08:37:58.349366+0000","last_undegraded":"2024-09-11T08:37:58.349366+0000","last_fullsized":"2024-09-11T08:37:58.349366+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:47:30.152568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350395+0000","last_change":"2024-09-11T08:37:55.060595+0000","last_active":"2024-09-11T08:37:58.350395+0000","last_peered":"2024-09-11T08:37:58.350395+0000","last_clean":"2024-09-11T08:37:58.350395+0000","last_became_active":"2024-09-11T08:37:55.060227+0000","last_became_peered":"2024-09-11T08:37:55.060227+0000","last_unstale":"2024-09-11T08:37:58.350395+0000","last_undegraded":"2024-09-11T08:37:58.350395+0000","last_fullsized":"2024-09-11T08:37:58.350395+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:39.197064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.063113+0000","last_change":"2024-09-11T08:37:51.271719+0000","last_active":"2024-09-11T08:37:59.063113+0000","last_peered":"2024-09-11T08:37:59.063113+0000","last_clean":"2024-09-11T08:37:59.063113+0000","last_became_active":"2024-09-11T08:37:51.270929+0000","last_became_peered":"2024-09-11T08:37:51.270929+0000","last_unstale":"2024-09-11T08:37:59.063113+0000","last_undegraded":"2024-09-11T08:37:59.063113+0000","last_fullsized":"2024-09-11T08:37:59.063113+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:05.724453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349614+0000","last_change":"2024-09-11T08:37:49.015790+0000","last_active":"2024-09-11T08:37:58.349614+0000","last_peered":"2024-09-11T08:37:58.349614+0000","last_clean":"2024-09-11T08:37:58.349614+0000","last_became_active":"2024-09-11T08:37:49.015534+0000","last_became_peered":"2024-09-11T08:37:49.015534+0000","last_unstale":"2024-09-11T08:37:58.349614+0000","last_undegraded":"2024-09-11T08:37:58.349614+0000","last_fullsized":"2024-09-11T08:37:58.349614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:33:08.397435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350589+0000","last_change":"2024-09-11T08:37:53.048372+0000","last_active":"2024-09-11T08:37:58.350589+0000","last_peered":"2024-09-11T08:37:58.350589+0000","last_clean":"2024-09-11T08:37:58.350589+0000","last_became_active":"2024-09-11T08:37:53.047912+0000","last_became_peered":"2024-09-11T08:37:53.047912+0000","last_unstale":"2024-09-11T08:37:58.350589+0000","last_undegraded":"2024-09-11T08:37:58.350589+0000","last_fullsized":"2024-09-11T08:37:58.350589+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:37:39.398681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349972+0000","last_change":"2024-09-11T08:37:55.050171+0000","last_active":"2024-09-11T08:37:58.349972+0000","last_peered":"2024-09-11T08:37:58.349972+0000","last_clean":"2024-09-11T08:37:58.349972+0000","last_became_active":"2024-09-11T08:37:55.049855+0000","last_became_peered":"2024-09-11T08:37:55.049855+0000","last_unstale":"2024-09-11T08:37:58.349972+0000","last_undegraded":"2024-09-11T08:37:58.349972+0000","last_fullsized":"2024-09-11T08:37:58.349972+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:19:13.422093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.481996+0000","last_change":"2024-09-11T08:37:49.018945+0000","last_active":"2024-09-11T08:37:58.481996+0000","last_peered":"2024-09-11T08:37:58.481996+0000","last_clean":"2024-09-11T08:37:58.481996+0000","last_became_active":"2024-09-11T08:37:49.018681+0000","last_became_peered":"2024-09-11T08:37:49.018681+0000","last_unstale":"2024-09-11T08:37:58.481996+0000","last_undegraded":"2024-09-11T08:37:58.481996+0000","last_fullsized":"2024-09-11T08:37:58.481996+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:02:12.287897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"59'6","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350690+0000","last_change":"2024-09-11T08:37:51.270626+0000","last_active":"2024-09-11T08:37:58.350690+0000","last_peered":"2024-09-11T08:37:58.350690+0000","last_clean":"2024-09-11T08:37:58.350690+0000","last_became_active":"2024-09-11T08:37:51.270339+0000","last_became_peered":"2024-09-11T08:37:51.270339+0000","last_unstale":"2024-09-11T08:37:58.350690+0000","last_undegraded":"2024-09-11T08:37:58.350690+0000","last_fullsized":"2024-09-11T08:37:58.350690+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:14.652394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.539084+0000","last_change":"2024-09-11T08:37:53.054957+0000","last_active":"2024-09-11T08:37:58.539084+0000","last_peered":"2024-09-11T08:37:58.539084+0000","last_clean":"2024-09-11T08:37:58.539084+0000","last_became_active":"2024-09-11T08:37:53.054669+0000","last_became_peered":"2024-09-11T08:37:53.054669+0000","last_unstale":"2024-09-11T08:37:58.539084+0000","last_undegraded":"2024-09-11T08:37:58.539084+0000","last_fullsized":"2024-09-11T08:37:58.539084+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:29.703816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523543+0000","last_change":"2024-09-11T08:37:55.436659+0000","last_active":"2024-09-11T08:37:59.523543+0000","last_peered":"2024-09-11T08:37:59.523543+0000","last_clean":"2024-09-11T08:37:59.523543+0000","last_became_active":"2024-09-11T08:37:55.436374+0000","last_became_peered":"2024-09-11T08:37:55.436374+0000","last_unstale":"2024-09-11T08:37:59.523543+0000","last_undegraded":"2024-09-11T08:37:59.523543+0000","last_fullsized":"2024-09-11T08:37:59.523543+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:27.765668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.351182+0000","last_change":"2024-09-11T08:37:51.271260+0000","last_active":"2024-09-11T08:37:58.351182+0000","last_peered":"2024-09-11T08:37:58.351182+0000","last_clean":"2024-09-11T08:37:58.351182+0000","last_became_active":"2024-09-11T08:37:51.270510+0000","last_became_peered":"2024-09-11T08:37:51.270510+0000","last_unstale":"2024-09-11T08:37:58.351182+0000","last_undegraded":"2024-09-11T08:37:58.351182+0000","last_fullsized":"2024-09-11T08:37:58.351182+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:30.154194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538745+0000","last_change":"2024-09-11T08:37:49.020927+0000","last_active":"2024-09-11T08:37:58.538745+0000","last_peered":"2024-09-11T08:37:58.538745+0000","last_clean":"2024-09-11T08:37:58.538745+0000","last_became_active":"2024-09-11T08:37:49.020332+0000","last_became_peered":"2024-09-11T08:37:49.020332+0000","last_unstale":"2024-09-11T08:37:58.538745+0000","last_undegraded":"2024-09-11T08:37:58.538745+0000","last_fullsized":"2024-09-11T08:37:58.538745+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:21:07.713308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183563+0000","last_change":"2024-09-11T08:37:53.058535+0000","last_active":"2024-09-11T08:39:07.183563+0000","last_peered":"2024-09-11T08:39:07.183563+0000","last_clean":"2024-09-11T08:39:07.183563+0000","last_became_active":"2024-09-11T08:37:53.057205+0000","last_became_peered":"2024-09-11T08:37:53.057205+0000","last_unstale":"2024-09-11T08:39:07.183563+0000","last_undegraded":"2024-09-11T08:39:07.183563+0000","last_fullsized":"2024-09-11T08:39:07.183563+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:11.171931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062833+0000","last_change":"2024-09-11T08:37:55.056184+0000","last_active":"2024-09-11T08:37:59.062833+0000","last_peered":"2024-09-11T08:37:59.062833+0000","last_clean":"2024-09-11T08:37:59.062833+0000","last_became_active":"2024-09-11T08:37:55.055890+0000","last_became_peered":"2024-09-11T08:37:55.055890+0000","last_unstale":"2024-09-11T08:37:59.062833+0000","last_undegraded":"2024-09-11T08:37:59.062833+0000","last_fullsized":"2024-09-11T08:37:59.062833+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:26.283164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.539062+0000","last_change":"2024-09-11T08:37:51.272114+0000","last_active":"2024-09-11T08:37:58.539062+0000","last_peered":"2024-09-11T08:37:58.539062+0000","last_clean":"2024-09-11T08:37:58.539062+0000","last_became_active":"2024-09-11T08:37:51.271808+0000","last_became_peered":"2024-09-11T08:37:51.271808+0000","last_unstale":"2024-09-11T08:37:58.539062+0000","last_undegraded":"2024-09-11T08:37:58.539062+0000","last_fullsized":"2024-09-11T08:37:58.539062+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:50:34.413005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062942+0000","last_change":"2024-09-11T08:37:49.018754+0000","last_active":"2024-09-11T08:37:59.062942+0000","last_peered":"2024-09-11T08:37:59.062942+0000","last_clean":"2024-09-11T08:37:59.062942+0000","last_became_active":"2024-09-11T08:37:49.018505+0000","last_became_peered":"2024-09-11T08:37:49.018505+0000","last_unstale":"2024-09-11T08:37:59.062942+0000","last_undegraded":"2024-09-11T08:37:59.062942+0000","last_fullsized":"2024-09-11T08:37:59.062942+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:46:07.978936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183477+0000","last_change":"2024-09-11T08:37:53.050592+0000","last_active":"2024-09-11T08:39:07.183477+0000","last_peered":"2024-09-11T08:39:07.183477+0000","last_clean":"2024-09-11T08:39:07.183477+0000","last_became_active":"2024-09-11T08:37:53.050271+0000","last_became_peered":"2024-09-11T08:37:53.050271+0000","last_unstale":"2024-09-11T08:39:07.183477+0000","last_undegraded":"2024-09-11T08:39:07.183477+0000","last_fullsized":"2024-09-11T08:39:07.183477+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:34.163018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349960+0000","last_change":"2024-09-11T08:37:55.048156+0000","last_active":"2024-09-11T08:37:58.349960+0000","last_peered":"2024-09-11T08:37:58.349960+0000","last_clean":"2024-09-11T08:37:58.349960+0000","last_became_active":"2024-09-11T08:37:55.047870+0000","last_became_peered":"2024-09-11T08:37:55.047870+0000","last_unstale":"2024-09-11T08:37:58.349960+0000","last_undegraded":"2024-09-11T08:37:58.349960+0000","last_fullsized":"2024-09-11T08:37:58.349960+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:54:51.443494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455096+0000","last_change":"2024-09-11T08:37:51.029933+0000","last_active":"2024-09-11T08:37:58.455096+0000","last_peered":"2024-09-11T08:37:58.455096+0000","last_clean":"2024-09-11T08:37:58.455096+0000","last_became_active":"2024-09-11T08:37:51.029591+0000","last_became_peered":"2024-09-11T08:37:51.029591+0000","last_unstale":"2024-09-11T08:37:58.455096+0000","last_undegraded":"2024-09-11T08:37:58.455096+0000","last_fullsized":"2024-09-11T08:37:58.455096+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:39:05.589753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.482117+0000","last_change":"2024-09-11T08:37:49.020452+0000","last_active":"2024-09-11T08:37:58.482117+0000","last_peered":"2024-09-11T08:37:58.482117+0000","last_clean":"2024-09-11T08:37:58.482117+0000","last_became_active":"2024-09-11T08:37:49.020204+0000","last_became_peered":"2024-09-11T08:37:49.020204+0000","last_unstale":"2024-09-11T08:37:58.482117+0000","last_undegraded":"2024-09-11T08:37:58.482117+0000","last_fullsized":"2024-09-11T08:37:58.482117+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:27:45.453726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350515+0000","last_change":"2024-09-11T08:37:53.045393+0000","last_active":"2024-09-11T08:37:58.350515+0000","last_peered":"2024-09-11T08:37:58.350515+0000","last_clean":"2024-09-11T08:37:58.350515+0000","last_became_active":"2024-09-11T08:37:53.045016+0000","last_became_peered":"2024-09-11T08:37:53.045016+0000","last_unstale":"2024-09-11T08:37:58.350515+0000","last_undegraded":"2024-09-11T08:37:58.350515+0000","last_fullsized":"2024-09-11T08:37:58.350515+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:29:50.859505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062085+0000","last_change":"2024-09-11T08:37:55.055359+0000","last_active":"2024-09-11T08:37:59.062085+0000","last_peered":"2024-09-11T08:37:59.062085+0000","last_clean":"2024-09-11T08:37:59.062085+0000","last_became_active":"2024-09-11T08:37:55.055068+0000","last_became_peered":"2024-09-11T08:37:55.055068+0000","last_unstale":"2024-09-11T08:37:59.062085+0000","last_undegraded":"2024-09-11T08:37:59.062085+0000","last_fullsized":"2024-09-11T08:37:59.062085+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:28:07.238727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349552+0000","last_change":"2024-09-11T08:37:51.030454+0000","last_active":"2024-09-11T08:37:58.349552+0000","last_peered":"2024-09-11T08:37:58.349552+0000","last_clean":"2024-09-11T08:37:58.349552+0000","last_became_active":"2024-09-11T08:37:51.030180+0000","last_became_peered":"2024-09-11T08:37:51.030180+0000","last_unstale":"2024-09-11T08:37:58.349552+0000","last_undegraded":"2024-09-11T08:37:58.349552+0000","last_fullsized":"2024-09-11T08:37:58.349552+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:49:45.812074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"52'1","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455815+0000","last_change":"2024-09-11T08:37:49.015695+0000","last_active":"2024-09-11T08:37:58.455815+0000","last_peered":"2024-09-11T08:37:58.455815+0000","last_clean":"2024-09-11T08:37:58.455815+0000","last_became_active":"2024-09-11T08:37:49.015387+0000","last_became_peered":"2024-09-11T08:37:49.015387+0000","last_unstale":"2024-09-11T08:37:58.455815+0000","last_undegraded":"2024-09-11T08:37:58.455815+0000","last_fullsized":"2024-09-11T08:37:58.455815+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:19:20.652872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455087+0000","last_change":"2024-09-11T08:37:53.053248+0000","last_active":"2024-09-11T08:37:58.455087+0000","last_peered":"2024-09-11T08:37:58.455087+0000","last_clean":"2024-09-11T08:37:58.455087+0000","last_became_active":"2024-09-11T08:37:53.052315+0000","last_became_peered":"2024-09-11T08:37:53.052315+0000","last_unstale":"2024-09-11T08:37:58.455087+0000","last_undegraded":"2024-09-11T08:37:58.455087+0000","last_fullsized":"2024-09-11T08:37:58.455087+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:39.083762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349756+0000","last_change":"2024-09-11T08:37:55.057967+0000","last_active":"2024-09-11T08:37:58.349756+0000","last_peered":"2024-09-11T08:37:58.349756+0000","last_clean":"2024-09-11T08:37:58.349756+0000","last_became_active":"2024-09-11T08:37:55.057615+0000","last_became_peered":"2024-09-11T08:37:55.057615+0000","last_unstale":"2024-09-11T08:37:58.349756+0000","last_undegraded":"2024-09-11T08:37:58.349756+0000","last_fullsized":"2024-09-11T08:37:58.349756+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:54.798827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"59'4","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349210+0000","last_change":"2024-09-11T08:37:51.036637+0000","last_active":"2024-09-11T08:37:58.349210+0000","last_peered":"2024-09-11T08:37:58.349210+0000","last_clean":"2024-09-11T08:37:58.349210+0000","last_became_active":"2024-09-11T08:37:51.035937+0000","last_became_peered":"2024-09-11T08:37:51.035937+0000","last_unstale":"2024-09-11T08:37:58.349210+0000","last_undegraded":"2024-09-11T08:37:58.349210+0000","last_fullsized":"2024-09-11T08:37:58.349210+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:45:16.268161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.539137+0000","last_change":"2024-09-11T08:37:49.022872+0000","last_active":"2024-09-11T08:37:58.539137+0000","last_peered":"2024-09-11T08:37:58.539137+0000","last_clean":"2024-09-11T08:37:58.539137+0000","last_became_active":"2024-09-11T08:37:49.022256+0000","last_became_peered":"2024-09-11T08:37:49.022256+0000","last_unstale":"2024-09-11T08:37:58.539137+0000","last_undegraded":"2024-09-11T08:37:58.539137+0000","last_fullsized":"2024-09-11T08:37:58.539137+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:02.890763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.481824+0000","last_change":"2024-09-11T08:37:53.054929+0000","last_active":"2024-09-11T08:37:58.481824+0000","last_peered":"2024-09-11T08:37:58.481824+0000","last_clean":"2024-09-11T08:37:58.481824+0000","last_became_active":"2024-09-11T08:37:53.054656+0000","last_became_peered":"2024-09-11T08:37:53.054656+0000","last_unstale":"2024-09-11T08:37:58.481824+0000","last_undegraded":"2024-09-11T08:37:58.481824+0000","last_fullsized":"2024-09-11T08:37:58.481824+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:48:22.340473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523068+0000","last_change":"2024-09-11T08:37:55.052961+0000","last_active":"2024-09-11T08:37:59.523068+0000","last_peered":"2024-09-11T08:37:59.523068+0000","last_clean":"2024-09-11T08:37:59.523068+0000","last_became_active":"2024-09-11T08:37:55.052681+0000","last_became_peered":"2024-09-11T08:37:55.052681+0000","last_unstale":"2024-09-11T08:37:59.523068+0000","last_undegraded":"2024-09-11T08:37:59.523068+0000","last_fullsized":"2024-09-11T08:37:59.523068+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:16:08.488202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350038+0000","last_change":"2024-09-11T08:37:51.271174+0000","last_active":"2024-09-11T08:37:58.350038+0000","last_peered":"2024-09-11T08:37:58.350038+0000","last_clean":"2024-09-11T08:37:58.350038+0000","last_became_active":"2024-09-11T08:37:51.270937+0000","last_became_peered":"2024-09-11T08:37:51.270937+0000","last_unstale":"2024-09-11T08:37:58.350038+0000","last_undegraded":"2024-09-11T08:37:58.350038+0000","last_fullsized":"2024-09-11T08:37:58.350038+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:01.796810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.482333+0000","last_change":"2024-09-11T08:37:49.018172+0000","last_active":"2024-09-11T08:37:58.482333+0000","last_peered":"2024-09-11T08:37:58.482333+0000","last_clean":"2024-09-11T08:37:58.482333+0000","last_became_active":"2024-09-11T08:37:49.017916+0000","last_became_peered":"2024-09-11T08:37:49.017916+0000","last_unstale":"2024-09-11T08:37:58.482333+0000","last_undegraded":"2024-09-11T08:37:58.482333+0000","last_fullsized":"2024-09-11T08:37:58.482333+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:01:32.572780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.522812+0000","last_change":"2024-09-11T08:37:53.049065+0000","last_active":"2024-09-11T08:37:59.522812+0000","last_peered":"2024-09-11T08:37:59.522812+0000","last_clean":"2024-09-11T08:37:59.522812+0000","last_became_active":"2024-09-11T08:37:53.048779+0000","last_became_peered":"2024-09-11T08:37:53.048779+0000","last_unstale":"2024-09-11T08:37:59.522812+0000","last_undegraded":"2024-09-11T08:37:59.522812+0000","last_fullsized":"2024-09-11T08:37:59.522812+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:12.708250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349149+0000","last_change":"2024-09-11T08:37:55.435290+0000","last_active":"2024-09-11T08:37:58.349149+0000","last_peered":"2024-09-11T08:37:58.349149+0000","last_clean":"2024-09-11T08:37:58.349149+0000","last_became_active":"2024-09-11T08:37:55.434752+0000","last_became_peered":"2024-09-11T08:37:55.434752+0000","last_unstale":"2024-09-11T08:37:58.349149+0000","last_undegraded":"2024-09-11T08:37:58.349149+0000","last_fullsized":"2024-09-11T08:37:58.349149+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:46:11.761668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523310+0000","last_change":"2024-09-11T08:37:55.052787+0000","last_active":"2024-09-11T08:37:59.523310+0000","last_peered":"2024-09-11T08:37:59.523310+0000","last_clean":"2024-09-11T08:37:59.523310+0000","last_became_active":"2024-09-11T08:37:55.052485+0000","last_became_peered":"2024-09-11T08:37:55.052485+0000","last_unstale":"2024-09-11T08:37:59.523310+0000","last_undegraded":"2024-09-11T08:37:59.523310+0000","last_fullsized":"2024-09-11T08:37:59.523310+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:06.627631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538562+0000","last_change":"2024-09-11T08:37:49.022999+0000","last_active":"2024-09-11T08:37:58.538562+0000","last_peered":"2024-09-11T08:37:58.538562+0000","last_clean":"2024-09-11T08:37:58.538562+0000","last_became_active":"2024-09-11T08:37:49.021669+0000","last_became_peered":"2024-09-11T08:37:49.021669+0000","last_unstale":"2024-09-11T08:37:58.538562+0000","last_undegraded":"2024-09-11T08:37:58.538562+0000","last_fullsized":"2024-09-11T08:37:58.538562+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:55.812373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.063039+0000","last_change":"2024-09-11T08:37:51.272436+0000","last_active":"2024-09-11T08:37:59.063039+0000","last_peered":"2024-09-11T08:37:59.063039+0000","last_clean":"2024-09-11T08:37:59.063039+0000","last_became_active":"2024-09-11T08:37:51.271729+0000","last_became_peered":"2024-09-11T08:37:51.271729+0000","last_unstale":"2024-09-11T08:37:59.063039+0000","last_undegraded":"2024-09-11T08:37:59.063039+0000","last_fullsized":"2024-09-11T08:37:59.063039+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:45:27.117726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"59'8","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183561+0000","last_change":"2024-09-11T08:37:53.056346+0000","last_active":"2024-09-11T08:39:07.183561+0000","last_peered":"2024-09-11T08:39:07.183561+0000","last_clean":"2024-09-11T08:39:07.183561+0000","last_became_active":"2024-09-11T08:37:53.055198+0000","last_became_peered":"2024-09-11T08:37:53.055198+0000","last_unstale":"2024-09-11T08:39:07.183561+0000","last_undegraded":"2024-09-11T08:39:07.183561+0000","last_fullsized":"2024-09-11T08:39:07.183561+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:05:49.517104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455140+0000","last_change":"2024-09-11T08:37:55.049768+0000","last_active":"2024-09-11T08:37:58.455140+0000","last_peered":"2024-09-11T08:37:58.455140+0000","last_clean":"2024-09-11T08:37:58.455140+0000","last_became_active":"2024-09-11T08:37:55.049485+0000","last_became_peered":"2024-09-11T08:37:55.049485+0000","last_unstale":"2024-09-11T08:37:58.455140+0000","last_undegraded":"2024-09-11T08:37:58.455140+0000","last_fullsized":"2024-09-11T08:37:58.455140+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:51:56.447519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350233+0000","last_change":"2024-09-11T08:37:49.024005+0000","last_active":"2024-09-11T08:37:58.350233+0000","last_peered":"2024-09-11T08:37:58.350233+0000","last_clean":"2024-09-11T08:37:58.350233+0000","last_became_active":"2024-09-11T08:37:49.023743+0000","last_became_peered":"2024-09-11T08:37:49.023743+0000","last_unstale":"2024-09-11T08:37:58.350233+0000","last_undegraded":"2024-09-11T08:37:58.350233+0000","last_fullsized":"2024-09-11T08:37:58.350233+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:11:57.613960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.482293+0000","last_change":"2024-09-11T08:37:51.033795+0000","last_active":"2024-09-11T08:37:58.482293+0000","last_peered":"2024-09-11T08:37:58.482293+0000","last_clean":"2024-09-11T08:37:58.482293+0000","last_became_active":"2024-09-11T08:37:51.033530+0000","last_became_peered":"2024-09-11T08:37:51.033530+0000","last_unstale":"2024-09-11T08:37:58.482293+0000","last_undegraded":"2024-09-11T08:37:58.482293+0000","last_fullsized":"2024-09-11T08:37:58.482293+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:34.205495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.063009+0000","last_change":"2024-09-11T08:37:53.049681+0000","last_active":"2024-09-11T08:37:59.063009+0000","last_peered":"2024-09-11T08:37:59.063009+0000","last_clean":"2024-09-11T08:37:59.063009+0000","last_became_active":"2024-09-11T08:37:53.049391+0000","last_became_peered":"2024-09-11T08:37:53.049391+0000","last_unstale":"2024-09-11T08:37:59.063009+0000","last_undegraded":"2024-09-11T08:37:59.063009+0000","last_fullsized":"2024-09-11T08:37:59.063009+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:17.798497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349749+0000","last_change":"2024-09-11T08:37:55.434487+0000","last_active":"2024-09-11T08:37:58.349749+0000","last_peered":"2024-09-11T08:37:58.349749+0000","last_clean":"2024-09-11T08:37:58.349749+0000","last_became_active":"2024-09-11T08:37:55.434193+0000","last_became_peered":"2024-09-11T08:37:55.434193+0000","last_unstale":"2024-09-11T08:37:58.349749+0000","last_undegraded":"2024-09-11T08:37:58.349749+0000","last_fullsized":"2024-09-11T08:37:58.349749+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:33:31.392445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.482091+0000","last_change":"2024-09-11T08:37:49.019897+0000","last_active":"2024-09-11T08:37:58.482091+0000","last_peered":"2024-09-11T08:37:58.482091+0000","last_clean":"2024-09-11T08:37:58.482091+0000","last_became_active":"2024-09-11T08:37:49.019610+0000","last_became_peered":"2024-09-11T08:37:49.019610+0000","last_unstale":"2024-09-11T08:37:58.482091+0000","last_undegraded":"2024-09-11T08:37:58.482091+0000","last_fullsized":"2024-09-11T08:37:58.482091+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:38:28.312052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350204+0000","last_change":"2024-09-11T08:37:51.033099+0000","last_active":"2024-09-11T08:37:58.350204+0000","last_peered":"2024-09-11T08:37:58.350204+0000","last_clean":"2024-09-11T08:37:58.350204+0000","last_became_active":"2024-09-11T08:37:51.032802+0000","last_became_peered":"2024-09-11T08:37:51.032802+0000","last_unstale":"2024-09-11T08:37:58.350204+0000","last_undegraded":"2024-09-11T08:37:58.350204+0000","last_fullsized":"2024-09-11T08:37:58.350204+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:49:43.471481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455432+0000","last_change":"2024-09-11T08:37:53.053004+0000","last_active":"2024-09-11T08:37:58.455432+0000","last_peered":"2024-09-11T08:37:58.455432+0000","last_clean":"2024-09-11T08:37:58.455432+0000","last_became_active":"2024-09-11T08:37:53.052732+0000","last_became_peered":"2024-09-11T08:37:53.052732+0000","last_unstale":"2024-09-11T08:37:58.455432+0000","last_undegraded":"2024-09-11T08:37:58.455432+0000","last_fullsized":"2024-09-11T08:37:58.455432+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:34:33.209197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350313+0000","last_change":"2024-09-11T08:37:55.434866+0000","last_active":"2024-09-11T08:37:58.350313+0000","last_peered":"2024-09-11T08:37:58.350313+0000","last_clean":"2024-09-11T08:37:58.350313+0000","last_became_active":"2024-09-11T08:37:55.434553+0000","last_became_peered":"2024-09-11T08:37:55.434553+0000","last_unstale":"2024-09-11T08:37:58.350313+0000","last_undegraded":"2024-09-11T08:37:58.350313+0000","last_fullsized":"2024-09-11T08:37:58.350313+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:39:45.243465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350708+0000","last_change":"2024-09-11T08:37:49.022591+0000","last_active":"2024-09-11T08:37:58.350708+0000","last_peered":"2024-09-11T08:37:58.350708+0000","last_clean":"2024-09-11T08:37:58.350708+0000","last_became_active":"2024-09-11T08:37:49.022314+0000","last_became_peered":"2024-09-11T08:37:49.022314+0000","last_unstale":"2024-09-11T08:37:58.350708+0000","last_undegraded":"2024-09-11T08:37:58.350708+0000","last_fullsized":"2024-09-11T08:37:58.350708+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:25.308957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350626+0000","last_change":"2024-09-11T08:37:51.034941+0000","last_active":"2024-09-11T08:37:58.350626+0000","last_peered":"2024-09-11T08:37:58.350626+0000","last_clean":"2024-09-11T08:37:58.350626+0000","last_became_active":"2024-09-11T08:37:51.034571+0000","last_became_peered":"2024-09-11T08:37:51.034571+0000","last_unstale":"2024-09-11T08:37:58.350626+0000","last_undegraded":"2024-09-11T08:37:58.350626+0000","last_fullsized":"2024-09-11T08:37:58.350626+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:00:32.482653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349778+0000","last_change":"2024-09-11T08:37:53.050703+0000","last_active":"2024-09-11T08:37:58.349778+0000","last_peered":"2024-09-11T08:37:58.349778+0000","last_clean":"2024-09-11T08:37:58.349778+0000","last_became_active":"2024-09-11T08:37:53.050429+0000","last_became_peered":"2024-09-11T08:37:53.050429+0000","last_unstale":"2024-09-11T08:37:58.349778+0000","last_undegraded":"2024-09-11T08:37:58.349778+0000","last_fullsized":"2024-09-11T08:37:58.349778+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:51:57.937403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430229,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298536,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446616,"statfs":{"total":95995035648,"available":95689334784,"internally_reserved":0,"allocated":1208320,"data_stored":822690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.50900000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.54900000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":10.023,"5min":10.023,"15min":10.023},"last":0.48599999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.52200000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.375}]},{"osd":5,"last update":"Wed Sep 11 08:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.40000000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.59099999999999997}]}]},{"osd":6,"up_from":42,"seq":180388626457,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298508,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446644,"statfs":{"total":95995035648,"available":95689363456,"internally_reserved":0,"allocated":1183744,"data_stored":821798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.60299999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed Sep 11 08:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.48799999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.629}]},{"osd":4,"last update":"Wed Sep 11 08:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.64500000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58299999999999996}]},{"osd":7,"last update":"Wed Sep 11 08:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71399999999999997}]}]},{"osd":5,"up_from":36,"seq":154618822686,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":598016,"data_stored":232253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.502,"15min":0.502},"min":{"1min":0.38100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63900000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.504,"15min":0.499},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72199999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed Sep 11 08:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.41499999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.501,"15min":0.504},"min":{"1min":0.38200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.63600000000000001,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.40799999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.32500000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66400000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52000000000000002,"15min":0.51900000000000002},"min":{"1min":0.373,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.38300000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65100000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.51200000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.379,"15min":0.375},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.40899999999999997},"min":{"1min":0.252,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.625}]},{"osd":6,"last update":"Wed Sep 11 08:38:14 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.60799999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:38:38 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.56299999999999994}]}]},{"osd":4,"up_from":30,"seq":128849018915,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297956,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447196,"statfs":{"total":95995035648,"available":95689928704,"internally_reserved":0,"allocated":614400,"data_stored":232113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64400000000000002,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.35499999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.65300000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.39800000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.30299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.38600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.623,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68400000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.55000000000000004,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.63800000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56899999999999995}]},{"osd":3,"last update":"Wed Sep 11 08:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.317,"5min":0.311,"15min":0.311},"max":{"1min":0.67400000000000004,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.39600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.71999999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.53100000000000003}]},{"osd":5,"last update":"Wed Sep 11 08:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.28100000000000003,"5min":0.253,"15min":0.253},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39700000000000002,"15min":0.39800000000000002},"min":{"1min":0.254,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63300000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.505}]},{"osd":6,"last update":"Wed Sep 11 08:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.67800000000000005}]},{"osd":7,"last update":"Wed Sep 11 08:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.63400000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182438,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297976,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447176,"statfs":{"total":95995035648,"available":95689908224,"internally_reserved":0,"allocated":634880,"data_stored":232826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.38300000000000001,"15min":0.39300000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.54500000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.379,"15min":0.374},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.53500000000000003,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.36799999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.27300000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.58999999999999997,"5min":0.621,"15min":0.621},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40300000000000002,"15min":0.41099999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52500000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.41699999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.41299999999999998,"15min":0.42599999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56299999999999994,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40999999999999998,"15min":0.41599999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.55000000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.40000000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55100000000000005,"15min":0.55400000000000005},"min":{"1min":0.39200000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.71799999999999997,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.39500000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71399999999999997,"5min":0.874,"15min":0.874},"last":0.55700000000000005}]},{"osd":5,"last update":"Wed Sep 11 08:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73899999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.498},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78100000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.63700000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:38:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.65100000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":598016,"data_stored":232116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:38:47 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.34200000000000003,"15min":0.33800000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.53800000000000003,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.307},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35699999999999998,"15min":0.34899999999999998},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.58599999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:38:47 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35499999999999998,"15min":0.34399999999999997},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.55100000000000005,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.83599999999999997,"15min":1.0389999999999999},"min":{"1min":0.248,"5min":0.246,"15min":0.246},"max":{"1min":0.629,"5min":15.186,"15min":15.186},"last":0.34999999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.41799999999999998,"15min":0.438},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.54400000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.42099999999999999,"15min":0.443},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.52100000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.38}]},{"osd":4,"last update":"Wed Sep 11 08:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74199999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40699999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76300000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.54400000000000004}]},{"osd":5,"last update":"Wed Sep 11 08:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.55000000000000004,"15min":0.55900000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71099999999999997,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82299999999999995,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.44600000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:38:42 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.60699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297988,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447164,"statfs":{"total":95995035648,"available":95689895936,"internally_reserved":0,"allocated":647168,"data_stored":233880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:38:23 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.32400000000000001,"15min":0.308},"min":{"1min":0.247,"5min":0.221,"15min":0.221},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.34699999999999998,"15min":0.32200000000000001},"min":{"1min":0.26400000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.55900000000000005,"5min":0.625,"15min":0.625},"last":0.51300000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:38:52 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34899999999999998,"15min":0.34300000000000003},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.49099999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.38,"15min":0.36799999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.52300000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:38:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.432,"15min":0.42499999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.38},"min":{"1min":0.30099999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.436}]},{"osd":4,"last update":"Wed Sep 11 08:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":3.746,"5min":3.746,"15min":3.746},"last":0.77100000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:38:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.75}]},{"osd":6,"last update":"Wed Sep 11 08:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.78300000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:38:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.81599999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298528,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446624,"statfs":{"total":95995035648,"available":95689342976,"internally_reserved":0,"allocated":1204224,"data_stored":822886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:38:27 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.313,"15min":0.29299999999999998},"min":{"1min":0.218,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.499,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.33100000000000002,"15min":0.307},"min":{"1min":0.249,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.26100000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35099999999999998,"15min":0.34300000000000003},"min":{"1min":0.23300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.379,"15min":0.371},"min":{"1min":0.245,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.32600000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:38:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52300000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.438,"15min":0.437},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.55500000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.35199999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:38:31 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55600000000000005}]},{"osd":5,"last update":"Wed Sep 11 08:38:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76000000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.626},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56899999999999995,"15min":0.57299999999999995},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85999999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.56899999999999995}]},{"osd":6,"last update":"Wed Sep 11 08:38:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.50600000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:38:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.54500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:39:14.769 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:39:15.017 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:15.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:15 smithi131 ceph-mon[29446]: from='client.24641 v1:172.21.15.131:0/2311596021' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:39:15.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:15 smithi131 ceph-mon[29446]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:15.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:15 smithi131 ceph-mon[37208]: from='client.24641 v1:172.21.15.131:0/2311596021' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:39:15.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:15 smithi131 ceph-mon[37208]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:15.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:15 smithi189 ceph-mon[32675]: from='client.24641 v1:172.21.15.131:0/2311596021' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:39:15.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:15 smithi189 ceph-mon[32675]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:16.829 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:39:16.831 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-11T08:39:17.485 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-11T08:39:16.215024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":928,"num_read_kb":785,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385372,"kb_used_data":6532,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575844,"statfs":{"total":767960285184,"available":765517664256,"internally_reserved":0,"allocated":6688768,"data_stored":3630562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002449"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349063+0000","last_change":"2024-09-11T08:37:55.434898+0000","last_active":"2024-09-11T08:37:58.349063+0000","last_peered":"2024-09-11T08:37:58.349063+0000","last_clean":"2024-09-11T08:37:58.349063+0000","last_became_active":"2024-09-11T08:37:55.434521+0000","last_became_peered":"2024-09-11T08:37:55.434521+0000","last_unstale":"2024-09-11T08:37:58.349063+0000","last_undegraded":"2024-09-11T08:37:58.349063+0000","last_fullsized":"2024-09-11T08:37:58.349063+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:48:40.076543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.063071+0000","last_change":"2024-09-11T08:37:49.019977+0000","last_active":"2024-09-11T08:37:59.063071+0000","last_peered":"2024-09-11T08:37:59.063071+0000","last_clean":"2024-09-11T08:37:59.063071+0000","last_became_active":"2024-09-11T08:37:49.019077+0000","last_became_peered":"2024-09-11T08:37:49.019077+0000","last_unstale":"2024-09-11T08:37:59.063071+0000","last_undegraded":"2024-09-11T08:37:59.063071+0000","last_fullsized":"2024-09-11T08:37:59.063071+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:11.665331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350332+0000","last_change":"2024-09-11T08:37:51.033534+0000","last_active":"2024-09-11T08:37:58.350332+0000","last_peered":"2024-09-11T08:37:58.350332+0000","last_clean":"2024-09-11T08:37:58.350332+0000","last_became_active":"2024-09-11T08:37:51.033263+0000","last_became_peered":"2024-09-11T08:37:51.033263+0000","last_unstale":"2024-09-11T08:37:58.350332+0000","last_undegraded":"2024-09-11T08:37:58.350332+0000","last_fullsized":"2024-09-11T08:37:58.350332+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:14:10.665290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349300+0000","last_change":"2024-09-11T08:37:53.048567+0000","last_active":"2024-09-11T08:37:58.349300+0000","last_peered":"2024-09-11T08:37:58.349300+0000","last_clean":"2024-09-11T08:37:58.349300+0000","last_became_active":"2024-09-11T08:37:53.048255+0000","last_became_peered":"2024-09-11T08:37:53.048255+0000","last_unstale":"2024-09-11T08:37:58.349300+0000","last_undegraded":"2024-09-11T08:37:58.349300+0000","last_fullsized":"2024-09-11T08:37:58.349300+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:27:48.550065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350393+0000","last_change":"2024-09-11T08:37:49.024131+0000","last_active":"2024-09-11T08:37:58.350393+0000","last_peered":"2024-09-11T08:37:58.350393+0000","last_clean":"2024-09-11T08:37:58.350393+0000","last_became_active":"2024-09-11T08:37:49.023836+0000","last_became_peered":"2024-09-11T08:37:49.023836+0000","last_unstale":"2024-09-11T08:37:58.350393+0000","last_undegraded":"2024-09-11T08:37:58.350393+0000","last_fullsized":"2024-09-11T08:37:58.350393+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:52:50.743128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349300+0000","last_change":"2024-09-11T08:37:51.033574+0000","last_active":"2024-09-11T08:37:58.349300+0000","last_peered":"2024-09-11T08:37:58.349300+0000","last_clean":"2024-09-11T08:37:58.349300+0000","last_became_active":"2024-09-11T08:37:51.033235+0000","last_became_peered":"2024-09-11T08:37:51.033235+0000","last_unstale":"2024-09-11T08:37:58.349300+0000","last_undegraded":"2024-09-11T08:37:58.349300+0000","last_fullsized":"2024-09-11T08:37:58.349300+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:33.844308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.454758+0000","last_change":"2024-09-11T08:37:53.052412+0000","last_active":"2024-09-11T08:37:58.454758+0000","last_peered":"2024-09-11T08:37:58.454758+0000","last_clean":"2024-09-11T08:37:58.454758+0000","last_became_active":"2024-09-11T08:37:53.051983+0000","last_became_peered":"2024-09-11T08:37:53.051983+0000","last_unstale":"2024-09-11T08:37:58.454758+0000","last_undegraded":"2024-09-11T08:37:58.454758+0000","last_fullsized":"2024-09-11T08:37:58.454758+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:10:20.565175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349417+0000","last_change":"2024-09-11T08:37:55.048153+0000","last_active":"2024-09-11T08:37:58.349417+0000","last_peered":"2024-09-11T08:37:58.349417+0000","last_clean":"2024-09-11T08:37:58.349417+0000","last_became_active":"2024-09-11T08:37:55.047646+0000","last_became_peered":"2024-09-11T08:37:55.047646+0000","last_unstale":"2024-09-11T08:37:58.349417+0000","last_undegraded":"2024-09-11T08:37:58.349417+0000","last_fullsized":"2024-09-11T08:37:58.349417+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:40.841075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523541+0000","last_change":"2024-09-11T08:37:49.024673+0000","last_active":"2024-09-11T08:37:59.523541+0000","last_peered":"2024-09-11T08:37:59.523541+0000","last_clean":"2024-09-11T08:37:59.523541+0000","last_became_active":"2024-09-11T08:37:49.024253+0000","last_became_peered":"2024-09-11T08:37:49.024253+0000","last_unstale":"2024-09-11T08:37:59.523541+0000","last_undegraded":"2024-09-11T08:37:59.523541+0000","last_fullsized":"2024-09-11T08:37:59.523541+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:33.729941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"59'5","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349521+0000","last_change":"2024-09-11T08:37:51.030719+0000","last_active":"2024-09-11T08:37:58.349521+0000","last_peered":"2024-09-11T08:37:58.349521+0000","last_clean":"2024-09-11T08:37:58.349521+0000","last_became_active":"2024-09-11T08:37:51.030370+0000","last_became_peered":"2024-09-11T08:37:51.030370+0000","last_unstale":"2024-09-11T08:37:58.349521+0000","last_undegraded":"2024-09-11T08:37:58.349521+0000","last_fullsized":"2024-09-11T08:37:58.349521+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:13:10.948458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523110+0000","last_change":"2024-09-11T08:37:53.045348+0000","last_active":"2024-09-11T08:37:59.523110+0000","last_peered":"2024-09-11T08:37:59.523110+0000","last_clean":"2024-09-11T08:37:59.523110+0000","last_became_active":"2024-09-11T08:37:53.044964+0000","last_became_peered":"2024-09-11T08:37:53.044964+0000","last_unstale":"2024-09-11T08:37:59.523110+0000","last_undegraded":"2024-09-11T08:37:59.523110+0000","last_fullsized":"2024-09-11T08:37:59.523110+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:22.901601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538636+0000","last_change":"2024-09-11T08:37:55.058413+0000","last_active":"2024-09-11T08:37:58.538636+0000","last_peered":"2024-09-11T08:37:58.538636+0000","last_clean":"2024-09-11T08:37:58.538636+0000","last_became_active":"2024-09-11T08:37:55.058030+0000","last_became_peered":"2024-09-11T08:37:55.058030+0000","last_unstale":"2024-09-11T08:37:58.538636+0000","last_undegraded":"2024-09-11T08:37:58.538636+0000","last_fullsized":"2024-09-11T08:37:58.538636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:12.469407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523468+0000","last_change":"2024-09-11T08:37:49.021837+0000","last_active":"2024-09-11T08:37:59.523468+0000","last_peered":"2024-09-11T08:37:59.523468+0000","last_clean":"2024-09-11T08:37:59.523468+0000","last_became_active":"2024-09-11T08:37:49.021579+0000","last_became_peered":"2024-09-11T08:37:49.021579+0000","last_unstale":"2024-09-11T08:37:59.523468+0000","last_undegraded":"2024-09-11T08:37:59.523468+0000","last_fullsized":"2024-09-11T08:37:59.523468+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:12:14.919864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349559+0000","last_change":"2024-09-11T08:37:51.032386+0000","last_active":"2024-09-11T08:37:58.349559+0000","last_peered":"2024-09-11T08:37:58.349559+0000","last_clean":"2024-09-11T08:37:58.349559+0000","last_became_active":"2024-09-11T08:37:51.032155+0000","last_became_peered":"2024-09-11T08:37:51.032155+0000","last_unstale":"2024-09-11T08:37:58.349559+0000","last_undegraded":"2024-09-11T08:37:58.349559+0000","last_fullsized":"2024-09-11T08:37:58.349559+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:51:52.890198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538795+0000","last_change":"2024-09-11T08:37:53.053926+0000","last_active":"2024-09-11T08:37:58.538795+0000","last_peered":"2024-09-11T08:37:58.538795+0000","last_clean":"2024-09-11T08:37:58.538795+0000","last_became_active":"2024-09-11T08:37:53.053645+0000","last_became_peered":"2024-09-11T08:37:53.053645+0000","last_unstale":"2024-09-11T08:37:58.538795+0000","last_undegraded":"2024-09-11T08:37:58.538795+0000","last_fullsized":"2024-09-11T08:37:58.538795+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:14:04.984953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062355+0000","last_change":"2024-09-11T08:37:55.054741+0000","last_active":"2024-09-11T08:37:59.062355+0000","last_peered":"2024-09-11T08:37:59.062355+0000","last_clean":"2024-09-11T08:37:59.062355+0000","last_became_active":"2024-09-11T08:37:55.054441+0000","last_became_peered":"2024-09-11T08:37:55.054441+0000","last_unstale":"2024-09-11T08:37:59.062355+0000","last_undegraded":"2024-09-11T08:37:59.062355+0000","last_fullsized":"2024-09-11T08:37:59.062355+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:04:13.643930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"59'17","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350004+0000","last_change":"2024-09-11T08:37:51.033463+0000","last_active":"2024-09-11T08:37:58.350004+0000","last_peered":"2024-09-11T08:37:58.350004+0000","last_clean":"2024-09-11T08:37:58.350004+0000","last_became_active":"2024-09-11T08:37:51.033153+0000","last_became_peered":"2024-09-11T08:37:51.033153+0000","last_unstale":"2024-09-11T08:37:58.350004+0000","last_undegraded":"2024-09-11T08:37:58.350004+0000","last_fullsized":"2024-09-11T08:37:58.350004+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:19:01.876473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523338+0000","last_change":"2024-09-11T08:37:49.022131+0000","last_active":"2024-09-11T08:37:59.523338+0000","last_peered":"2024-09-11T08:37:59.523338+0000","last_clean":"2024-09-11T08:37:59.523338+0000","last_became_active":"2024-09-11T08:37:49.021654+0000","last_became_peered":"2024-09-11T08:37:49.021654+0000","last_unstale":"2024-09-11T08:37:59.523338+0000","last_undegraded":"2024-09-11T08:37:59.523338+0000","last_fullsized":"2024-09-11T08:37:59.523338+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:33.224730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455373+0000","last_change":"2024-09-11T08:37:53.052918+0000","last_active":"2024-09-11T08:37:58.455373+0000","last_peered":"2024-09-11T08:37:58.455373+0000","last_clean":"2024-09-11T08:37:58.455373+0000","last_became_active":"2024-09-11T08:37:53.052603+0000","last_became_peered":"2024-09-11T08:37:53.052603+0000","last_unstale":"2024-09-11T08:37:58.455373+0000","last_undegraded":"2024-09-11T08:37:58.455373+0000","last_fullsized":"2024-09-11T08:37:58.455373+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:32:30.639347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349540+0000","last_change":"2024-09-11T08:37:55.058670+0000","last_active":"2024-09-11T08:37:58.349540+0000","last_peered":"2024-09-11T08:37:58.349540+0000","last_clean":"2024-09-11T08:37:58.349540+0000","last_became_active":"2024-09-11T08:37:55.058385+0000","last_became_peered":"2024-09-11T08:37:55.058385+0000","last_unstale":"2024-09-11T08:37:58.349540+0000","last_undegraded":"2024-09-11T08:37:58.349540+0000","last_fullsized":"2024-09-11T08:37:58.349540+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:52:46.878375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349729+0000","last_change":"2024-09-11T08:37:51.030783+0000","last_active":"2024-09-11T08:37:58.349729+0000","last_peered":"2024-09-11T08:37:58.349729+0000","last_clean":"2024-09-11T08:37:58.349729+0000","last_became_active":"2024-09-11T08:37:51.030507+0000","last_became_peered":"2024-09-11T08:37:51.030507+0000","last_unstale":"2024-09-11T08:37:58.349729+0000","last_undegraded":"2024-09-11T08:37:58.349729+0000","last_fullsized":"2024-09-11T08:37:58.349729+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:31.188722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"52'1","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455470+0000","last_change":"2024-09-11T08:37:49.021595+0000","last_active":"2024-09-11T08:37:58.455470+0000","last_peered":"2024-09-11T08:37:58.455470+0000","last_clean":"2024-09-11T08:37:58.455470+0000","last_became_active":"2024-09-11T08:37:49.021339+0000","last_became_peered":"2024-09-11T08:37:49.021339+0000","last_unstale":"2024-09-11T08:37:58.455470+0000","last_undegraded":"2024-09-11T08:37:58.455470+0000","last_fullsized":"2024-09-11T08:37:58.455470+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:11:06.396322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183424+0000","last_change":"2024-09-11T08:37:53.054096+0000","last_active":"2024-09-11T08:39:07.183424+0000","last_peered":"2024-09-11T08:39:07.183424+0000","last_clean":"2024-09-11T08:39:07.183424+0000","last_became_active":"2024-09-11T08:37:53.053782+0000","last_became_peered":"2024-09-11T08:37:53.053782+0000","last_unstale":"2024-09-11T08:39:07.183424+0000","last_undegraded":"2024-09-11T08:39:07.183424+0000","last_fullsized":"2024-09-11T08:39:07.183424+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:57:54.554544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349748+0000","last_change":"2024-09-11T08:37:55.048149+0000","last_active":"2024-09-11T08:37:58.349748+0000","last_peered":"2024-09-11T08:37:58.349748+0000","last_clean":"2024-09-11T08:37:58.349748+0000","last_became_active":"2024-09-11T08:37:55.047884+0000","last_became_peered":"2024-09-11T08:37:55.047884+0000","last_unstale":"2024-09-11T08:37:58.349748+0000","last_undegraded":"2024-09-11T08:37:58.349748+0000","last_fullsized":"2024-09-11T08:37:58.349748+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:47:34.276764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455866+0000","last_change":"2024-09-11T08:37:51.032131+0000","last_active":"2024-09-11T08:37:58.455866+0000","last_peered":"2024-09-11T08:37:58.455866+0000","last_clean":"2024-09-11T08:37:58.455866+0000","last_became_active":"2024-09-11T08:37:51.031859+0000","last_became_peered":"2024-09-11T08:37:51.031859+0000","last_unstale":"2024-09-11T08:37:58.455866+0000","last_undegraded":"2024-09-11T08:37:58.455866+0000","last_fullsized":"2024-09-11T08:37:58.455866+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:36:12.907878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.454941+0000","last_change":"2024-09-11T08:37:49.019359+0000","last_active":"2024-09-11T08:37:58.454941+0000","last_peered":"2024-09-11T08:37:58.454941+0000","last_clean":"2024-09-11T08:37:58.454941+0000","last_became_active":"2024-09-11T08:37:49.018945+0000","last_became_peered":"2024-09-11T08:37:49.018945+0000","last_unstale":"2024-09-11T08:37:58.454941+0000","last_undegraded":"2024-09-11T08:37:58.454941+0000","last_fullsized":"2024-09-11T08:37:58.454941+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:16:48.101619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183569+0000","last_change":"2024-09-11T08:37:53.048189+0000","last_active":"2024-09-11T08:39:07.183569+0000","last_peered":"2024-09-11T08:39:07.183569+0000","last_clean":"2024-09-11T08:39:07.183569+0000","last_became_active":"2024-09-11T08:37:53.047910+0000","last_became_peered":"2024-09-11T08:37:53.047910+0000","last_unstale":"2024-09-11T08:39:07.183569+0000","last_undegraded":"2024-09-11T08:39:07.183569+0000","last_fullsized":"2024-09-11T08:39:07.183569+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:47.431633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538934+0000","last_change":"2024-09-11T08:37:55.054848+0000","last_active":"2024-09-11T08:37:58.538934+0000","last_peered":"2024-09-11T08:37:58.538934+0000","last_clean":"2024-09-11T08:37:58.538934+0000","last_became_active":"2024-09-11T08:37:55.054584+0000","last_became_peered":"2024-09-11T08:37:55.054584+0000","last_unstale":"2024-09-11T08:37:58.538934+0000","last_undegraded":"2024-09-11T08:37:58.538934+0000","last_fullsized":"2024-09-11T08:37:58.538934+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:45:09.313154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349820+0000","last_change":"2024-09-11T08:37:51.033616+0000","last_active":"2024-09-11T08:37:58.349820+0000","last_peered":"2024-09-11T08:37:58.349820+0000","last_clean":"2024-09-11T08:37:58.349820+0000","last_became_active":"2024-09-11T08:37:51.033269+0000","last_became_peered":"2024-09-11T08:37:51.033269+0000","last_unstale":"2024-09-11T08:37:58.349820+0000","last_undegraded":"2024-09-11T08:37:58.349820+0000","last_fullsized":"2024-09-11T08:37:58.349820+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:40.380938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.522991+0000","last_change":"2024-09-11T08:37:49.020669+0000","last_active":"2024-09-11T08:37:59.522991+0000","last_peered":"2024-09-11T08:37:59.522991+0000","last_clean":"2024-09-11T08:37:59.522991+0000","last_became_active":"2024-09-11T08:37:49.020094+0000","last_became_peered":"2024-09-11T08:37:49.020094+0000","last_unstale":"2024-09-11T08:37:59.522991+0000","last_undegraded":"2024-09-11T08:37:59.522991+0000","last_fullsized":"2024-09-11T08:37:59.522991+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:08:46.495559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.539165+0000","last_change":"2024-09-11T08:37:53.052207+0000","last_active":"2024-09-11T08:37:58.539165+0000","last_peered":"2024-09-11T08:37:58.539165+0000","last_clean":"2024-09-11T08:37:58.539165+0000","last_became_active":"2024-09-11T08:37:53.051934+0000","last_became_peered":"2024-09-11T08:37:53.051934+0000","last_unstale":"2024-09-11T08:37:58.539165+0000","last_undegraded":"2024-09-11T08:37:58.539165+0000","last_fullsized":"2024-09-11T08:37:58.539165+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:31:12.635418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350830+0000","last_change":"2024-09-11T08:37:55.436076+0000","last_active":"2024-09-11T08:37:58.350830+0000","last_peered":"2024-09-11T08:37:58.350830+0000","last_clean":"2024-09-11T08:37:58.350830+0000","last_became_active":"2024-09-11T08:37:55.435653+0000","last_became_peered":"2024-09-11T08:37:55.435653+0000","last_unstale":"2024-09-11T08:37:58.350830+0000","last_undegraded":"2024-09-11T08:37:58.350830+0000","last_fullsized":"2024-09-11T08:37:58.350830+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:21:15.501305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"59'14","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349384+0000","last_change":"2024-09-11T08:37:51.029239+0000","last_active":"2024-09-11T08:37:58.349384+0000","last_peered":"2024-09-11T08:37:58.349384+0000","last_clean":"2024-09-11T08:37:58.349384+0000","last_became_active":"2024-09-11T08:37:51.028910+0000","last_became_peered":"2024-09-11T08:37:51.028910+0000","last_unstale":"2024-09-11T08:37:58.349384+0000","last_undegraded":"2024-09-11T08:37:58.349384+0000","last_fullsized":"2024-09-11T08:37:58.349384+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:47.188353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.482338+0000","last_change":"2024-09-11T08:37:49.019139+0000","last_active":"2024-09-11T08:37:58.482338+0000","last_peered":"2024-09-11T08:37:58.482338+0000","last_clean":"2024-09-11T08:37:58.482338+0000","last_became_active":"2024-09-11T08:37:49.018914+0000","last_became_peered":"2024-09-11T08:37:49.018914+0000","last_unstale":"2024-09-11T08:37:58.482338+0000","last_undegraded":"2024-09-11T08:37:58.482338+0000","last_fullsized":"2024-09-11T08:37:58.482338+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:29:13.244505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"59'8","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183481+0000","last_change":"2024-09-11T08:37:53.047443+0000","last_active":"2024-09-11T08:39:07.183481+0000","last_peered":"2024-09-11T08:39:07.183481+0000","last_clean":"2024-09-11T08:39:07.183481+0000","last_became_active":"2024-09-11T08:37:53.047124+0000","last_became_peered":"2024-09-11T08:37:53.047124+0000","last_unstale":"2024-09-11T08:39:07.183481+0000","last_undegraded":"2024-09-11T08:39:07.183481+0000","last_fullsized":"2024-09-11T08:39:07.183481+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:43.958314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523197+0000","last_change":"2024-09-11T08:37:55.433815+0000","last_active":"2024-09-11T08:37:59.523197+0000","last_peered":"2024-09-11T08:37:59.523197+0000","last_clean":"2024-09-11T08:37:59.523197+0000","last_became_active":"2024-09-11T08:37:55.433529+0000","last_became_peered":"2024-09-11T08:37:55.433529+0000","last_unstale":"2024-09-11T08:37:59.523197+0000","last_undegraded":"2024-09-11T08:37:59.523197+0000","last_fullsized":"2024-09-11T08:37:59.523197+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:23:54.418499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"59'18","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349109+0000","last_change":"2024-09-11T08:37:51.271316+0000","last_active":"2024-09-11T08:37:58.349109+0000","last_peered":"2024-09-11T08:37:58.349109+0000","last_clean":"2024-09-11T08:37:58.349109+0000","last_became_active":"2024-09-11T08:37:51.271057+0000","last_became_peered":"2024-09-11T08:37:51.271057+0000","last_unstale":"2024-09-11T08:37:58.349109+0000","last_undegraded":"2024-09-11T08:37:58.349109+0000","last_fullsized":"2024-09-11T08:37:58.349109+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:30:24.674369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455544+0000","last_change":"2024-09-11T08:37:49.014008+0000","last_active":"2024-09-11T08:37:58.455544+0000","last_peered":"2024-09-11T08:37:58.455544+0000","last_clean":"2024-09-11T08:37:58.455544+0000","last_became_active":"2024-09-11T08:37:49.013731+0000","last_became_peered":"2024-09-11T08:37:49.013731+0000","last_unstale":"2024-09-11T08:37:58.455544+0000","last_undegraded":"2024-09-11T08:37:58.455544+0000","last_fullsized":"2024-09-11T08:37:58.455544+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:28:20.967515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349316+0000","last_change":"2024-09-11T08:37:53.047434+0000","last_active":"2024-09-11T08:37:58.349316+0000","last_peered":"2024-09-11T08:37:58.349316+0000","last_clean":"2024-09-11T08:37:58.349316+0000","last_became_active":"2024-09-11T08:37:53.047158+0000","last_became_peered":"2024-09-11T08:37:53.047158+0000","last_unstale":"2024-09-11T08:37:58.349316+0000","last_undegraded":"2024-09-11T08:37:58.349316+0000","last_fullsized":"2024-09-11T08:37:58.349316+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:30:36.115849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349524+0000","last_change":"2024-09-11T08:37:55.054722+0000","last_active":"2024-09-11T08:37:58.349524+0000","last_peered":"2024-09-11T08:37:58.349524+0000","last_clean":"2024-09-11T08:37:58.349524+0000","last_became_active":"2024-09-11T08:37:55.054470+0000","last_became_peered":"2024-09-11T08:37:55.054470+0000","last_unstale":"2024-09-11T08:37:58.349524+0000","last_undegraded":"2024-09-11T08:37:58.349524+0000","last_fullsized":"2024-09-11T08:37:58.349524+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:54:52.735680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"59'19","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062777+0000","last_change":"2024-09-11T08:37:51.271190+0000","last_active":"2024-09-11T08:37:59.062777+0000","last_peered":"2024-09-11T08:37:59.062777+0000","last_clean":"2024-09-11T08:37:59.062777+0000","last_became_active":"2024-09-11T08:37:51.270341+0000","last_became_peered":"2024-09-11T08:37:51.270341+0000","last_unstale":"2024-09-11T08:37:59.062777+0000","last_undegraded":"2024-09-11T08:37:59.062777+0000","last_fullsized":"2024-09-11T08:37:59.062777+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:43.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523593+0000","last_change":"2024-09-11T08:37:49.023943+0000","last_active":"2024-09-11T08:37:59.523593+0000","last_peered":"2024-09-11T08:37:59.523593+0000","last_clean":"2024-09-11T08:37:59.523593+0000","last_became_active":"2024-09-11T08:37:49.022862+0000","last_became_peered":"2024-09-11T08:37:49.022862+0000","last_unstale":"2024-09-11T08:37:59.523593+0000","last_undegraded":"2024-09-11T08:37:59.523593+0000","last_fullsized":"2024-09-11T08:37:59.523593+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:44:14.646330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.481869+0000","last_change":"2024-09-11T08:37:53.053668+0000","last_active":"2024-09-11T08:37:58.481869+0000","last_peered":"2024-09-11T08:37:58.481869+0000","last_clean":"2024-09-11T08:37:58.481869+0000","last_became_active":"2024-09-11T08:37:53.053389+0000","last_became_peered":"2024-09-11T08:37:53.053389+0000","last_unstale":"2024-09-11T08:37:58.481869+0000","last_undegraded":"2024-09-11T08:37:58.481869+0000","last_fullsized":"2024-09-11T08:37:58.481869+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:47:26.099011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.454704+0000","last_change":"2024-09-11T08:37:55.433983+0000","last_active":"2024-09-11T08:37:58.454704+0000","last_peered":"2024-09-11T08:37:58.454704+0000","last_clean":"2024-09-11T08:37:58.454704+0000","last_became_active":"2024-09-11T08:37:55.433578+0000","last_became_peered":"2024-09-11T08:37:55.433578+0000","last_unstale":"2024-09-11T08:37:58.454704+0000","last_undegraded":"2024-09-11T08:37:58.454704+0000","last_fullsized":"2024-09-11T08:37:58.454704+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:05:42.370642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455406+0000","last_change":"2024-09-11T08:37:51.029311+0000","last_active":"2024-09-11T08:37:58.455406+0000","last_peered":"2024-09-11T08:37:58.455406+0000","last_clean":"2024-09-11T08:37:58.455406+0000","last_became_active":"2024-09-11T08:37:51.028561+0000","last_became_peered":"2024-09-11T08:37:51.028561+0000","last_unstale":"2024-09-11T08:37:58.455406+0000","last_undegraded":"2024-09-11T08:37:58.455406+0000","last_fullsized":"2024-09-11T08:37:58.455406+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:30:42.923055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455397+0000","last_change":"2024-09-11T08:37:49.019254+0000","last_active":"2024-09-11T08:37:58.455397+0000","last_peered":"2024-09-11T08:37:58.455397+0000","last_clean":"2024-09-11T08:37:58.455397+0000","last_became_active":"2024-09-11T08:37:49.018991+0000","last_became_peered":"2024-09-11T08:37:49.018991+0000","last_unstale":"2024-09-11T08:37:58.455397+0000","last_undegraded":"2024-09-11T08:37:58.455397+0000","last_fullsized":"2024-09-11T08:37:58.455397+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:27.169174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183431+0000","last_change":"2024-09-11T08:37:53.049824+0000","last_active":"2024-09-11T08:39:07.183431+0000","last_peered":"2024-09-11T08:39:07.183431+0000","last_clean":"2024-09-11T08:39:07.183431+0000","last_became_active":"2024-09-11T08:37:53.049537+0000","last_became_peered":"2024-09-11T08:37:53.049537+0000","last_unstale":"2024-09-11T08:39:07.183431+0000","last_undegraded":"2024-09-11T08:39:07.183431+0000","last_fullsized":"2024-09-11T08:39:07.183431+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:05.414755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062162+0000","last_change":"2024-09-11T08:37:55.057894+0000","last_active":"2024-09-11T08:37:59.062162+0000","last_peered":"2024-09-11T08:37:59.062162+0000","last_clean":"2024-09-11T08:37:59.062162+0000","last_became_active":"2024-09-11T08:37:55.057573+0000","last_became_peered":"2024-09-11T08:37:55.057573+0000","last_unstale":"2024-09-11T08:37:59.062162+0000","last_undegraded":"2024-09-11T08:37:59.062162+0000","last_fullsized":"2024-09-11T08:37:59.062162+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:25.123202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"59'28","reported_seq":81,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183534+0000","last_change":"2024-09-11T08:37:51.031243+0000","last_active":"2024-09-11T08:39:07.183534+0000","last_peered":"2024-09-11T08:39:07.183534+0000","last_clean":"2024-09-11T08:39:07.183534+0000","last_became_active":"2024-09-11T08:37:51.030859+0000","last_became_peered":"2024-09-11T08:37:51.030859+0000","last_unstale":"2024-09-11T08:39:07.183534+0000","last_undegraded":"2024-09-11T08:39:07.183534+0000","last_fullsized":"2024-09-11T08:39:07.183534+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:50:08.622543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538770+0000","last_change":"2024-09-11T08:37:49.020429+0000","last_active":"2024-09-11T08:37:58.538770+0000","last_peered":"2024-09-11T08:37:58.538770+0000","last_clean":"2024-09-11T08:37:58.538770+0000","last_became_active":"2024-09-11T08:37:49.020167+0000","last_became_peered":"2024-09-11T08:37:49.020167+0000","last_unstale":"2024-09-11T08:37:58.538770+0000","last_undegraded":"2024-09-11T08:37:58.538770+0000","last_fullsized":"2024-09-11T08:37:58.538770+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:39.265497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062928+0000","last_change":"2024-09-11T08:37:53.047856+0000","last_active":"2024-09-11T08:37:59.062928+0000","last_peered":"2024-09-11T08:37:59.062928+0000","last_clean":"2024-09-11T08:37:59.062928+0000","last_became_active":"2024-09-11T08:37:53.046447+0000","last_became_peered":"2024-09-11T08:37:53.046447+0000","last_unstale":"2024-09-11T08:37:59.062928+0000","last_undegraded":"2024-09-11T08:37:59.062928+0000","last_fullsized":"2024-09-11T08:37:59.062928+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:32:51.819175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350129+0000","last_change":"2024-09-11T08:37:55.055772+0000","last_active":"2024-09-11T08:37:58.350129+0000","last_peered":"2024-09-11T08:37:58.350129+0000","last_clean":"2024-09-11T08:37:58.350129+0000","last_became_active":"2024-09-11T08:37:55.055503+0000","last_became_peered":"2024-09-11T08:37:55.055503+0000","last_unstale":"2024-09-11T08:37:58.350129+0000","last_undegraded":"2024-09-11T08:37:58.350129+0000","last_fullsized":"2024-09-11T08:37:58.350129+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:05.095131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'13","reported_seq":43,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455916+0000","last_change":"2024-09-11T08:37:51.034069+0000","last_active":"2024-09-11T08:37:58.455916+0000","last_peered":"2024-09-11T08:37:58.455916+0000","last_clean":"2024-09-11T08:37:58.455916+0000","last_became_active":"2024-09-11T08:37:51.033633+0000","last_became_peered":"2024-09-11T08:37:51.033633+0000","last_unstale":"2024-09-11T08:37:58.455916+0000","last_undegraded":"2024-09-11T08:37:58.455916+0000","last_fullsized":"2024-09-11T08:37:58.455916+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:20:42.157777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"60'5","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:11.112089+0000","last_change":"2024-09-11T08:37:53.040217+0000","last_active":"2024-09-11T08:39:11.112089+0000","last_peered":"2024-09-11T08:39:11.112089+0000","last_clean":"2024-09-11T08:39:11.112089+0000","last_became_active":"2024-09-11T08:37:49.016198+0000","last_became_peered":"2024-09-11T08:37:49.016198+0000","last_unstale":"2024-09-11T08:39:11.112089+0000","last_undegraded":"2024-09-11T08:39:11.112089+0000","last_fullsized":"2024-09-11T08:39:11.112089+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:08:19.801628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017901740000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349878+0000","last_change":"2024-09-11T08:37:49.020128+0000","last_active":"2024-09-11T08:37:58.349878+0000","last_peered":"2024-09-11T08:37:58.349878+0000","last_clean":"2024-09-11T08:37:58.349878+0000","last_became_active":"2024-09-11T08:37:49.019785+0000","last_became_peered":"2024-09-11T08:37:49.019785+0000","last_unstale":"2024-09-11T08:37:58.349878+0000","last_undegraded":"2024-09-11T08:37:58.349878+0000","last_fullsized":"2024-09-11T08:37:58.349878+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:37:24.004661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349739+0000","last_change":"2024-09-11T08:37:53.054033+0000","last_active":"2024-09-11T08:37:58.349739+0000","last_peered":"2024-09-11T08:37:58.349739+0000","last_clean":"2024-09-11T08:37:58.349739+0000","last_became_active":"2024-09-11T08:37:53.053718+0000","last_became_peered":"2024-09-11T08:37:53.053718+0000","last_unstale":"2024-09-11T08:37:58.349739+0000","last_undegraded":"2024-09-11T08:37:58.349739+0000","last_fullsized":"2024-09-11T08:37:58.349739+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:00:23.513884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523363+0000","last_change":"2024-09-11T08:37:55.433912+0000","last_active":"2024-09-11T08:37:59.523363+0000","last_peered":"2024-09-11T08:37:59.523363+0000","last_clean":"2024-09-11T08:37:59.523363+0000","last_became_active":"2024-09-11T08:37:55.433510+0000","last_became_peered":"2024-09-11T08:37:55.433510+0000","last_unstale":"2024-09-11T08:37:59.523363+0000","last_undegraded":"2024-09-11T08:37:59.523363+0000","last_fullsized":"2024-09-11T08:37:59.523363+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:28:34.418337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"59'12","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062241+0000","last_change":"2024-09-11T08:37:51.037580+0000","last_active":"2024-09-11T08:37:59.062241+0000","last_peered":"2024-09-11T08:37:59.062241+0000","last_clean":"2024-09-11T08:37:59.062241+0000","last_became_active":"2024-09-11T08:37:51.037288+0000","last_became_peered":"2024-09-11T08:37:51.037288+0000","last_unstale":"2024-09-11T08:37:59.062241+0000","last_undegraded":"2024-09-11T08:37:59.062241+0000","last_fullsized":"2024-09-11T08:37:59.062241+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:48:14.566225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"54'1","reported_seq":24,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062619+0000","last_change":"2024-09-11T08:37:53.039112+0000","last_active":"2024-09-11T08:37:59.062619+0000","last_peered":"2024-09-11T08:37:59.062619+0000","last_clean":"2024-09-11T08:37:59.062619+0000","last_became_active":"2024-09-11T08:37:49.020745+0000","last_became_peered":"2024-09-11T08:37:49.020745+0000","last_unstale":"2024-09-11T08:37:59.062619+0000","last_undegraded":"2024-09-11T08:37:59.062619+0000","last_fullsized":"2024-09-11T08:37:59.062619+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:01.595693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061342700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523051+0000","last_change":"2024-09-11T08:37:49.023769+0000","last_active":"2024-09-11T08:37:59.523051+0000","last_peered":"2024-09-11T08:37:59.523051+0000","last_clean":"2024-09-11T08:37:59.523051+0000","last_became_active":"2024-09-11T08:37:49.022720+0000","last_became_peered":"2024-09-11T08:37:49.022720+0000","last_unstale":"2024-09-11T08:37:59.523051+0000","last_undegraded":"2024-09-11T08:37:59.523051+0000","last_fullsized":"2024-09-11T08:37:59.523051+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:17:32.792408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.539165+0000","last_change":"2024-09-11T08:37:53.054982+0000","last_active":"2024-09-11T08:37:58.539165+0000","last_peered":"2024-09-11T08:37:58.539165+0000","last_clean":"2024-09-11T08:37:58.539165+0000","last_became_active":"2024-09-11T08:37:53.054729+0000","last_became_peered":"2024-09-11T08:37:53.054729+0000","last_unstale":"2024-09-11T08:37:58.539165+0000","last_undegraded":"2024-09-11T08:37:58.539165+0000","last_fullsized":"2024-09-11T08:37:58.539165+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:04:47.381949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455025+0000","last_change":"2024-09-11T08:37:55.056675+0000","last_active":"2024-09-11T08:37:58.455025+0000","last_peered":"2024-09-11T08:37:58.455025+0000","last_clean":"2024-09-11T08:37:58.455025+0000","last_became_active":"2024-09-11T08:37:55.056339+0000","last_became_peered":"2024-09-11T08:37:55.056339+0000","last_unstale":"2024-09-11T08:37:58.455025+0000","last_undegraded":"2024-09-11T08:37:58.455025+0000","last_fullsized":"2024-09-11T08:37:58.455025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:34:22.793615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"59'14","reported_seq":38,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.482069+0000","last_change":"2024-09-11T08:37:51.271898+0000","last_active":"2024-09-11T08:37:58.482069+0000","last_peered":"2024-09-11T08:37:58.482069+0000","last_clean":"2024-09-11T08:37:58.482069+0000","last_became_active":"2024-09-11T08:37:51.271313+0000","last_became_peered":"2024-09-11T08:37:51.271313+0000","last_unstale":"2024-09-11T08:37:58.482069+0000","last_undegraded":"2024-09-11T08:37:58.482069+0000","last_fullsized":"2024-09-11T08:37:58.482069+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:17:03.912213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'2","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349550+0000","last_change":"2024-09-11T08:37:53.044478+0000","last_active":"2024-09-11T08:37:58.349550+0000","last_peered":"2024-09-11T08:37:58.349550+0000","last_clean":"2024-09-11T08:37:58.349550+0000","last_became_active":"2024-09-11T08:37:49.021133+0000","last_became_peered":"2024-09-11T08:37:49.021133+0000","last_unstale":"2024-09-11T08:37:58.349550+0000","last_undegraded":"2024-09-11T08:37:58.349550+0000","last_fullsized":"2024-09-11T08:37:58.349550+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:20:00.991673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026546719999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538944+0000","last_change":"2024-09-11T08:37:49.023418+0000","last_active":"2024-09-11T08:37:58.538944+0000","last_peered":"2024-09-11T08:37:58.538944+0000","last_clean":"2024-09-11T08:37:58.538944+0000","last_became_active":"2024-09-11T08:37:49.023138+0000","last_became_peered":"2024-09-11T08:37:49.023138+0000","last_unstale":"2024-09-11T08:37:58.538944+0000","last_undegraded":"2024-09-11T08:37:58.538944+0000","last_fullsized":"2024-09-11T08:37:58.538944+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:06.531998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"61'79","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:38:28.254556+0000","last_change":"2024-09-11T08:37:39.200133+0000","last_active":"2024-09-11T08:38:28.254556+0000","last_peered":"2024-09-11T08:38:28.254556+0000","last_clean":"2024-09-11T08:38:28.254556+0000","last_became_active":"2024-09-11T08:37:39.193681+0000","last_became_peered":"2024-09-11T08:37:39.193681+0000","last_unstale":"2024-09-11T08:38:28.254556+0000","last_undegraded":"2024-09-11T08:38:28.254556+0000","last_fullsized":"2024-09-11T08:38:28.254556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:35:44.055988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:35:44.055988+0000","last_clean_scrub_stamp":"2024-09-11T08:35:44.055988+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:03:15.848922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523327+0000","last_change":"2024-09-11T08:37:53.050902+0000","last_active":"2024-09-11T08:37:59.523327+0000","last_peered":"2024-09-11T08:37:59.523327+0000","last_clean":"2024-09-11T08:37:59.523327+0000","last_became_active":"2024-09-11T08:37:53.050575+0000","last_became_peered":"2024-09-11T08:37:53.050575+0000","last_unstale":"2024-09-11T08:37:59.523327+0000","last_undegraded":"2024-09-11T08:37:59.523327+0000","last_fullsized":"2024-09-11T08:37:59.523327+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:13:07.312142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.539241+0000","last_change":"2024-09-11T08:37:55.059161+0000","last_active":"2024-09-11T08:37:58.539241+0000","last_peered":"2024-09-11T08:37:58.539241+0000","last_clean":"2024-09-11T08:37:58.539241+0000","last_became_active":"2024-09-11T08:37:55.058853+0000","last_became_peered":"2024-09-11T08:37:55.058853+0000","last_unstale":"2024-09-11T08:37:58.539241+0000","last_undegraded":"2024-09-11T08:37:58.539241+0000","last_fullsized":"2024-09-11T08:37:58.539241+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:44.113253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"59'19","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.482002+0000","last_change":"2024-09-11T08:37:51.271006+0000","last_active":"2024-09-11T08:37:58.482002+0000","last_peered":"2024-09-11T08:37:58.482002+0000","last_clean":"2024-09-11T08:37:58.482002+0000","last_became_active":"2024-09-11T08:37:51.270631+0000","last_became_peered":"2024-09-11T08:37:51.270631+0000","last_unstale":"2024-09-11T08:37:58.482002+0000","last_undegraded":"2024-09-11T08:37:58.482002+0000","last_fullsized":"2024-09-11T08:37:58.482002+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:49:53.798485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350354+0000","last_change":"2024-09-11T08:37:49.018768+0000","last_active":"2024-09-11T08:37:58.350354+0000","last_peered":"2024-09-11T08:37:58.350354+0000","last_clean":"2024-09-11T08:37:58.350354+0000","last_became_active":"2024-09-11T08:37:49.017506+0000","last_became_peered":"2024-09-11T08:37:49.017506+0000","last_unstale":"2024-09-11T08:37:58.350354+0000","last_undegraded":"2024-09-11T08:37:58.350354+0000","last_fullsized":"2024-09-11T08:37:58.350354+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:18.561931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349411+0000","last_change":"2024-09-11T08:37:53.051776+0000","last_active":"2024-09-11T08:37:58.349411+0000","last_peered":"2024-09-11T08:37:58.349411+0000","last_clean":"2024-09-11T08:37:58.349411+0000","last_became_active":"2024-09-11T08:37:53.051476+0000","last_became_peered":"2024-09-11T08:37:53.051476+0000","last_unstale":"2024-09-11T08:37:58.349411+0000","last_undegraded":"2024-09-11T08:37:58.349411+0000","last_fullsized":"2024-09-11T08:37:58.349411+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:21:25.949454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523120+0000","last_change":"2024-09-11T08:37:55.058963+0000","last_active":"2024-09-11T08:37:59.523120+0000","last_peered":"2024-09-11T08:37:59.523120+0000","last_clean":"2024-09-11T08:37:59.523120+0000","last_became_active":"2024-09-11T08:37:55.058576+0000","last_became_peered":"2024-09-11T08:37:55.058576+0000","last_unstale":"2024-09-11T08:37:59.523120+0000","last_undegraded":"2024-09-11T08:37:59.523120+0000","last_fullsized":"2024-09-11T08:37:59.523120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:03:31.703591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.061984+0000","last_change":"2024-09-11T08:37:51.033565+0000","last_active":"2024-09-11T08:37:59.061984+0000","last_peered":"2024-09-11T08:37:59.061984+0000","last_clean":"2024-09-11T08:37:59.061984+0000","last_became_active":"2024-09-11T08:37:51.033264+0000","last_became_peered":"2024-09-11T08:37:51.033264+0000","last_unstale":"2024-09-11T08:37:59.061984+0000","last_undegraded":"2024-09-11T08:37:59.061984+0000","last_fullsized":"2024-09-11T08:37:59.061984+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:53.014927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455776+0000","last_change":"2024-09-11T08:37:49.017286+0000","last_active":"2024-09-11T08:37:58.455776+0000","last_peered":"2024-09-11T08:37:58.455776+0000","last_clean":"2024-09-11T08:37:58.455776+0000","last_became_active":"2024-09-11T08:37:49.017025+0000","last_became_peered":"2024-09-11T08:37:49.017025+0000","last_unstale":"2024-09-11T08:37:58.455776+0000","last_undegraded":"2024-09-11T08:37:58.455776+0000","last_fullsized":"2024-09-11T08:37:58.455776+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:39:26.463336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349344+0000","last_change":"2024-09-11T08:37:53.052060+0000","last_active":"2024-09-11T08:37:58.349344+0000","last_peered":"2024-09-11T08:37:58.349344+0000","last_clean":"2024-09-11T08:37:58.349344+0000","last_became_active":"2024-09-11T08:37:53.051738+0000","last_became_peered":"2024-09-11T08:37:53.051738+0000","last_unstale":"2024-09-11T08:37:58.349344+0000","last_undegraded":"2024-09-11T08:37:58.349344+0000","last_fullsized":"2024-09-11T08:37:58.349344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:02.789274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062493+0000","last_change":"2024-09-11T08:37:55.054762+0000","last_active":"2024-09-11T08:37:59.062493+0000","last_peered":"2024-09-11T08:37:59.062493+0000","last_clean":"2024-09-11T08:37:59.062493+0000","last_became_active":"2024-09-11T08:37:55.054348+0000","last_became_peered":"2024-09-11T08:37:55.054348+0000","last_unstale":"2024-09-11T08:37:59.062493+0000","last_undegraded":"2024-09-11T08:37:59.062493+0000","last_fullsized":"2024-09-11T08:37:59.062493+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:39.743624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538526+0000","last_change":"2024-09-11T08:37:51.271344+0000","last_active":"2024-09-11T08:37:58.538526+0000","last_peered":"2024-09-11T08:37:58.538526+0000","last_clean":"2024-09-11T08:37:58.538526+0000","last_became_active":"2024-09-11T08:37:51.271042+0000","last_became_peered":"2024-09-11T08:37:51.271042+0000","last_unstale":"2024-09-11T08:37:58.538526+0000","last_undegraded":"2024-09-11T08:37:58.538526+0000","last_fullsized":"2024-09-11T08:37:58.538526+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:13.715865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349936+0000","last_change":"2024-09-11T08:37:49.018660+0000","last_active":"2024-09-11T08:37:58.349936+0000","last_peered":"2024-09-11T08:37:58.349936+0000","last_clean":"2024-09-11T08:37:58.349936+0000","last_became_active":"2024-09-11T08:37:49.018423+0000","last_became_peered":"2024-09-11T08:37:49.018423+0000","last_unstale":"2024-09-11T08:37:58.349936+0000","last_undegraded":"2024-09-11T08:37:58.349936+0000","last_fullsized":"2024-09-11T08:37:58.349936+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:02:26.645297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183576+0000","last_change":"2024-09-11T08:37:53.049055+0000","last_active":"2024-09-11T08:39:07.183576+0000","last_peered":"2024-09-11T08:39:07.183576+0000","last_clean":"2024-09-11T08:39:07.183576+0000","last_became_active":"2024-09-11T08:37:53.048765+0000","last_became_peered":"2024-09-11T08:37:53.048765+0000","last_unstale":"2024-09-11T08:39:07.183576+0000","last_undegraded":"2024-09-11T08:39:07.183576+0000","last_fullsized":"2024-09-11T08:39:07.183576+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:56:21.957087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538610+0000","last_change":"2024-09-11T08:37:55.433918+0000","last_active":"2024-09-11T08:37:58.538610+0000","last_peered":"2024-09-11T08:37:58.538610+0000","last_clean":"2024-09-11T08:37:58.538610+0000","last_became_active":"2024-09-11T08:37:55.433544+0000","last_became_peered":"2024-09-11T08:37:55.433544+0000","last_unstale":"2024-09-11T08:37:58.538610+0000","last_undegraded":"2024-09-11T08:37:58.538610+0000","last_fullsized":"2024-09-11T08:37:58.538610+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:44:48.320347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349354+0000","last_change":"2024-09-11T08:37:51.033723+0000","last_active":"2024-09-11T08:37:58.349354+0000","last_peered":"2024-09-11T08:37:58.349354+0000","last_clean":"2024-09-11T08:37:58.349354+0000","last_became_active":"2024-09-11T08:37:51.033358+0000","last_became_peered":"2024-09-11T08:37:51.033358+0000","last_unstale":"2024-09-11T08:37:58.349354+0000","last_undegraded":"2024-09-11T08:37:58.349354+0000","last_fullsized":"2024-09-11T08:37:58.349354+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:57:03.991746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"52'2","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350091+0000","last_change":"2024-09-11T08:37:49.019809+0000","last_active":"2024-09-11T08:37:58.350091+0000","last_peered":"2024-09-11T08:37:58.350091+0000","last_clean":"2024-09-11T08:37:58.350091+0000","last_became_active":"2024-09-11T08:37:49.019553+0000","last_became_peered":"2024-09-11T08:37:49.019553+0000","last_unstale":"2024-09-11T08:37:58.350091+0000","last_undegraded":"2024-09-11T08:37:58.350091+0000","last_fullsized":"2024-09-11T08:37:58.350091+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:12:17.646393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349366+0000","last_change":"2024-09-11T08:37:53.051008+0000","last_active":"2024-09-11T08:37:58.349366+0000","last_peered":"2024-09-11T08:37:58.349366+0000","last_clean":"2024-09-11T08:37:58.349366+0000","last_became_active":"2024-09-11T08:37:53.050733+0000","last_became_peered":"2024-09-11T08:37:53.050733+0000","last_unstale":"2024-09-11T08:37:58.349366+0000","last_undegraded":"2024-09-11T08:37:58.349366+0000","last_fullsized":"2024-09-11T08:37:58.349366+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:47:30.152568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350395+0000","last_change":"2024-09-11T08:37:55.060595+0000","last_active":"2024-09-11T08:37:58.350395+0000","last_peered":"2024-09-11T08:37:58.350395+0000","last_clean":"2024-09-11T08:37:58.350395+0000","last_became_active":"2024-09-11T08:37:55.060227+0000","last_became_peered":"2024-09-11T08:37:55.060227+0000","last_unstale":"2024-09-11T08:37:58.350395+0000","last_undegraded":"2024-09-11T08:37:58.350395+0000","last_fullsized":"2024-09-11T08:37:58.350395+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:39.197064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.063113+0000","last_change":"2024-09-11T08:37:51.271719+0000","last_active":"2024-09-11T08:37:59.063113+0000","last_peered":"2024-09-11T08:37:59.063113+0000","last_clean":"2024-09-11T08:37:59.063113+0000","last_became_active":"2024-09-11T08:37:51.270929+0000","last_became_peered":"2024-09-11T08:37:51.270929+0000","last_unstale":"2024-09-11T08:37:59.063113+0000","last_undegraded":"2024-09-11T08:37:59.063113+0000","last_fullsized":"2024-09-11T08:37:59.063113+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:05.724453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349614+0000","last_change":"2024-09-11T08:37:49.015790+0000","last_active":"2024-09-11T08:37:58.349614+0000","last_peered":"2024-09-11T08:37:58.349614+0000","last_clean":"2024-09-11T08:37:58.349614+0000","last_became_active":"2024-09-11T08:37:49.015534+0000","last_became_peered":"2024-09-11T08:37:49.015534+0000","last_unstale":"2024-09-11T08:37:58.349614+0000","last_undegraded":"2024-09-11T08:37:58.349614+0000","last_fullsized":"2024-09-11T08:37:58.349614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:33:08.397435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350589+0000","last_change":"2024-09-11T08:37:53.048372+0000","last_active":"2024-09-11T08:37:58.350589+0000","last_peered":"2024-09-11T08:37:58.350589+0000","last_clean":"2024-09-11T08:37:58.350589+0000","last_became_active":"2024-09-11T08:37:53.047912+0000","last_became_peered":"2024-09-11T08:37:53.047912+0000","last_unstale":"2024-09-11T08:37:58.350589+0000","last_undegraded":"2024-09-11T08:37:58.350589+0000","last_fullsized":"2024-09-11T08:37:58.350589+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:37:39.398681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349972+0000","last_change":"2024-09-11T08:37:55.050171+0000","last_active":"2024-09-11T08:37:58.349972+0000","last_peered":"2024-09-11T08:37:58.349972+0000","last_clean":"2024-09-11T08:37:58.349972+0000","last_became_active":"2024-09-11T08:37:55.049855+0000","last_became_peered":"2024-09-11T08:37:55.049855+0000","last_unstale":"2024-09-11T08:37:58.349972+0000","last_undegraded":"2024-09-11T08:37:58.349972+0000","last_fullsized":"2024-09-11T08:37:58.349972+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:19:13.422093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.481996+0000","last_change":"2024-09-11T08:37:49.018945+0000","last_active":"2024-09-11T08:37:58.481996+0000","last_peered":"2024-09-11T08:37:58.481996+0000","last_clean":"2024-09-11T08:37:58.481996+0000","last_became_active":"2024-09-11T08:37:49.018681+0000","last_became_peered":"2024-09-11T08:37:49.018681+0000","last_unstale":"2024-09-11T08:37:58.481996+0000","last_undegraded":"2024-09-11T08:37:58.481996+0000","last_fullsized":"2024-09-11T08:37:58.481996+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:02:12.287897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"59'6","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350690+0000","last_change":"2024-09-11T08:37:51.270626+0000","last_active":"2024-09-11T08:37:58.350690+0000","last_peered":"2024-09-11T08:37:58.350690+0000","last_clean":"2024-09-11T08:37:58.350690+0000","last_became_active":"2024-09-11T08:37:51.270339+0000","last_became_peered":"2024-09-11T08:37:51.270339+0000","last_unstale":"2024-09-11T08:37:58.350690+0000","last_undegraded":"2024-09-11T08:37:58.350690+0000","last_fullsized":"2024-09-11T08:37:58.350690+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:14.652394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.539084+0000","last_change":"2024-09-11T08:37:53.054957+0000","last_active":"2024-09-11T08:37:58.539084+0000","last_peered":"2024-09-11T08:37:58.539084+0000","last_clean":"2024-09-11T08:37:58.539084+0000","last_became_active":"2024-09-11T08:37:53.054669+0000","last_became_peered":"2024-09-11T08:37:53.054669+0000","last_unstale":"2024-09-11T08:37:58.539084+0000","last_undegraded":"2024-09-11T08:37:58.539084+0000","last_fullsized":"2024-09-11T08:37:58.539084+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:29.703816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523543+0000","last_change":"2024-09-11T08:37:55.436659+0000","last_active":"2024-09-11T08:37:59.523543+0000","last_peered":"2024-09-11T08:37:59.523543+0000","last_clean":"2024-09-11T08:37:59.523543+0000","last_became_active":"2024-09-11T08:37:55.436374+0000","last_became_peered":"2024-09-11T08:37:55.436374+0000","last_unstale":"2024-09-11T08:37:59.523543+0000","last_undegraded":"2024-09-11T08:37:59.523543+0000","last_fullsized":"2024-09-11T08:37:59.523543+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:27.765668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.351182+0000","last_change":"2024-09-11T08:37:51.271260+0000","last_active":"2024-09-11T08:37:58.351182+0000","last_peered":"2024-09-11T08:37:58.351182+0000","last_clean":"2024-09-11T08:37:58.351182+0000","last_became_active":"2024-09-11T08:37:51.270510+0000","last_became_peered":"2024-09-11T08:37:51.270510+0000","last_unstale":"2024-09-11T08:37:58.351182+0000","last_undegraded":"2024-09-11T08:37:58.351182+0000","last_fullsized":"2024-09-11T08:37:58.351182+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:30.154194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538745+0000","last_change":"2024-09-11T08:37:49.020927+0000","last_active":"2024-09-11T08:37:58.538745+0000","last_peered":"2024-09-11T08:37:58.538745+0000","last_clean":"2024-09-11T08:37:58.538745+0000","last_became_active":"2024-09-11T08:37:49.020332+0000","last_became_peered":"2024-09-11T08:37:49.020332+0000","last_unstale":"2024-09-11T08:37:58.538745+0000","last_undegraded":"2024-09-11T08:37:58.538745+0000","last_fullsized":"2024-09-11T08:37:58.538745+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:21:07.713308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183563+0000","last_change":"2024-09-11T08:37:53.058535+0000","last_active":"2024-09-11T08:39:07.183563+0000","last_peered":"2024-09-11T08:39:07.183563+0000","last_clean":"2024-09-11T08:39:07.183563+0000","last_became_active":"2024-09-11T08:37:53.057205+0000","last_became_peered":"2024-09-11T08:37:53.057205+0000","last_unstale":"2024-09-11T08:39:07.183563+0000","last_undegraded":"2024-09-11T08:39:07.183563+0000","last_fullsized":"2024-09-11T08:39:07.183563+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:11.171931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062833+0000","last_change":"2024-09-11T08:37:55.056184+0000","last_active":"2024-09-11T08:37:59.062833+0000","last_peered":"2024-09-11T08:37:59.062833+0000","last_clean":"2024-09-11T08:37:59.062833+0000","last_became_active":"2024-09-11T08:37:55.055890+0000","last_became_peered":"2024-09-11T08:37:55.055890+0000","last_unstale":"2024-09-11T08:37:59.062833+0000","last_undegraded":"2024-09-11T08:37:59.062833+0000","last_fullsized":"2024-09-11T08:37:59.062833+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:26.283164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.539062+0000","last_change":"2024-09-11T08:37:51.272114+0000","last_active":"2024-09-11T08:37:58.539062+0000","last_peered":"2024-09-11T08:37:58.539062+0000","last_clean":"2024-09-11T08:37:58.539062+0000","last_became_active":"2024-09-11T08:37:51.271808+0000","last_became_peered":"2024-09-11T08:37:51.271808+0000","last_unstale":"2024-09-11T08:37:58.539062+0000","last_undegraded":"2024-09-11T08:37:58.539062+0000","last_fullsized":"2024-09-11T08:37:58.539062+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:50:34.413005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062942+0000","last_change":"2024-09-11T08:37:49.018754+0000","last_active":"2024-09-11T08:37:59.062942+0000","last_peered":"2024-09-11T08:37:59.062942+0000","last_clean":"2024-09-11T08:37:59.062942+0000","last_became_active":"2024-09-11T08:37:49.018505+0000","last_became_peered":"2024-09-11T08:37:49.018505+0000","last_unstale":"2024-09-11T08:37:59.062942+0000","last_undegraded":"2024-09-11T08:37:59.062942+0000","last_fullsized":"2024-09-11T08:37:59.062942+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:46:07.978936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183477+0000","last_change":"2024-09-11T08:37:53.050592+0000","last_active":"2024-09-11T08:39:07.183477+0000","last_peered":"2024-09-11T08:39:07.183477+0000","last_clean":"2024-09-11T08:39:07.183477+0000","last_became_active":"2024-09-11T08:37:53.050271+0000","last_became_peered":"2024-09-11T08:37:53.050271+0000","last_unstale":"2024-09-11T08:39:07.183477+0000","last_undegraded":"2024-09-11T08:39:07.183477+0000","last_fullsized":"2024-09-11T08:39:07.183477+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:34.163018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349960+0000","last_change":"2024-09-11T08:37:55.048156+0000","last_active":"2024-09-11T08:37:58.349960+0000","last_peered":"2024-09-11T08:37:58.349960+0000","last_clean":"2024-09-11T08:37:58.349960+0000","last_became_active":"2024-09-11T08:37:55.047870+0000","last_became_peered":"2024-09-11T08:37:55.047870+0000","last_unstale":"2024-09-11T08:37:58.349960+0000","last_undegraded":"2024-09-11T08:37:58.349960+0000","last_fullsized":"2024-09-11T08:37:58.349960+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:54:51.443494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455096+0000","last_change":"2024-09-11T08:37:51.029933+0000","last_active":"2024-09-11T08:37:58.455096+0000","last_peered":"2024-09-11T08:37:58.455096+0000","last_clean":"2024-09-11T08:37:58.455096+0000","last_became_active":"2024-09-11T08:37:51.029591+0000","last_became_peered":"2024-09-11T08:37:51.029591+0000","last_unstale":"2024-09-11T08:37:58.455096+0000","last_undegraded":"2024-09-11T08:37:58.455096+0000","last_fullsized":"2024-09-11T08:37:58.455096+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:39:05.589753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.482117+0000","last_change":"2024-09-11T08:37:49.020452+0000","last_active":"2024-09-11T08:37:58.482117+0000","last_peered":"2024-09-11T08:37:58.482117+0000","last_clean":"2024-09-11T08:37:58.482117+0000","last_became_active":"2024-09-11T08:37:49.020204+0000","last_became_peered":"2024-09-11T08:37:49.020204+0000","last_unstale":"2024-09-11T08:37:58.482117+0000","last_undegraded":"2024-09-11T08:37:58.482117+0000","last_fullsized":"2024-09-11T08:37:58.482117+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:27:45.453726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350515+0000","last_change":"2024-09-11T08:37:53.045393+0000","last_active":"2024-09-11T08:37:58.350515+0000","last_peered":"2024-09-11T08:37:58.350515+0000","last_clean":"2024-09-11T08:37:58.350515+0000","last_became_active":"2024-09-11T08:37:53.045016+0000","last_became_peered":"2024-09-11T08:37:53.045016+0000","last_unstale":"2024-09-11T08:37:58.350515+0000","last_undegraded":"2024-09-11T08:37:58.350515+0000","last_fullsized":"2024-09-11T08:37:58.350515+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:29:50.859505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.062085+0000","last_change":"2024-09-11T08:37:55.055359+0000","last_active":"2024-09-11T08:37:59.062085+0000","last_peered":"2024-09-11T08:37:59.062085+0000","last_clean":"2024-09-11T08:37:59.062085+0000","last_became_active":"2024-09-11T08:37:55.055068+0000","last_became_peered":"2024-09-11T08:37:55.055068+0000","last_unstale":"2024-09-11T08:37:59.062085+0000","last_undegraded":"2024-09-11T08:37:59.062085+0000","last_fullsized":"2024-09-11T08:37:59.062085+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:28:07.238727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349552+0000","last_change":"2024-09-11T08:37:51.030454+0000","last_active":"2024-09-11T08:37:58.349552+0000","last_peered":"2024-09-11T08:37:58.349552+0000","last_clean":"2024-09-11T08:37:58.349552+0000","last_became_active":"2024-09-11T08:37:51.030180+0000","last_became_peered":"2024-09-11T08:37:51.030180+0000","last_unstale":"2024-09-11T08:37:58.349552+0000","last_undegraded":"2024-09-11T08:37:58.349552+0000","last_fullsized":"2024-09-11T08:37:58.349552+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:49:45.812074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"52'1","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455815+0000","last_change":"2024-09-11T08:37:49.015695+0000","last_active":"2024-09-11T08:37:58.455815+0000","last_peered":"2024-09-11T08:37:58.455815+0000","last_clean":"2024-09-11T08:37:58.455815+0000","last_became_active":"2024-09-11T08:37:49.015387+0000","last_became_peered":"2024-09-11T08:37:49.015387+0000","last_unstale":"2024-09-11T08:37:58.455815+0000","last_undegraded":"2024-09-11T08:37:58.455815+0000","last_fullsized":"2024-09-11T08:37:58.455815+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:19:20.652872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455087+0000","last_change":"2024-09-11T08:37:53.053248+0000","last_active":"2024-09-11T08:37:58.455087+0000","last_peered":"2024-09-11T08:37:58.455087+0000","last_clean":"2024-09-11T08:37:58.455087+0000","last_became_active":"2024-09-11T08:37:53.052315+0000","last_became_peered":"2024-09-11T08:37:53.052315+0000","last_unstale":"2024-09-11T08:37:58.455087+0000","last_undegraded":"2024-09-11T08:37:58.455087+0000","last_fullsized":"2024-09-11T08:37:58.455087+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:39.083762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349756+0000","last_change":"2024-09-11T08:37:55.057967+0000","last_active":"2024-09-11T08:37:58.349756+0000","last_peered":"2024-09-11T08:37:58.349756+0000","last_clean":"2024-09-11T08:37:58.349756+0000","last_became_active":"2024-09-11T08:37:55.057615+0000","last_became_peered":"2024-09-11T08:37:55.057615+0000","last_unstale":"2024-09-11T08:37:58.349756+0000","last_undegraded":"2024-09-11T08:37:58.349756+0000","last_fullsized":"2024-09-11T08:37:58.349756+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:54.798827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"59'4","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349210+0000","last_change":"2024-09-11T08:37:51.036637+0000","last_active":"2024-09-11T08:37:58.349210+0000","last_peered":"2024-09-11T08:37:58.349210+0000","last_clean":"2024-09-11T08:37:58.349210+0000","last_became_active":"2024-09-11T08:37:51.035937+0000","last_became_peered":"2024-09-11T08:37:51.035937+0000","last_unstale":"2024-09-11T08:37:58.349210+0000","last_undegraded":"2024-09-11T08:37:58.349210+0000","last_fullsized":"2024-09-11T08:37:58.349210+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:45:16.268161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.539137+0000","last_change":"2024-09-11T08:37:49.022872+0000","last_active":"2024-09-11T08:37:58.539137+0000","last_peered":"2024-09-11T08:37:58.539137+0000","last_clean":"2024-09-11T08:37:58.539137+0000","last_became_active":"2024-09-11T08:37:49.022256+0000","last_became_peered":"2024-09-11T08:37:49.022256+0000","last_unstale":"2024-09-11T08:37:58.539137+0000","last_undegraded":"2024-09-11T08:37:58.539137+0000","last_fullsized":"2024-09-11T08:37:58.539137+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:02.890763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.481824+0000","last_change":"2024-09-11T08:37:53.054929+0000","last_active":"2024-09-11T08:37:58.481824+0000","last_peered":"2024-09-11T08:37:58.481824+0000","last_clean":"2024-09-11T08:37:58.481824+0000","last_became_active":"2024-09-11T08:37:53.054656+0000","last_became_peered":"2024-09-11T08:37:53.054656+0000","last_unstale":"2024-09-11T08:37:58.481824+0000","last_undegraded":"2024-09-11T08:37:58.481824+0000","last_fullsized":"2024-09-11T08:37:58.481824+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:48:22.340473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523068+0000","last_change":"2024-09-11T08:37:55.052961+0000","last_active":"2024-09-11T08:37:59.523068+0000","last_peered":"2024-09-11T08:37:59.523068+0000","last_clean":"2024-09-11T08:37:59.523068+0000","last_became_active":"2024-09-11T08:37:55.052681+0000","last_became_peered":"2024-09-11T08:37:55.052681+0000","last_unstale":"2024-09-11T08:37:59.523068+0000","last_undegraded":"2024-09-11T08:37:59.523068+0000","last_fullsized":"2024-09-11T08:37:59.523068+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:16:08.488202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350038+0000","last_change":"2024-09-11T08:37:51.271174+0000","last_active":"2024-09-11T08:37:58.350038+0000","last_peered":"2024-09-11T08:37:58.350038+0000","last_clean":"2024-09-11T08:37:58.350038+0000","last_became_active":"2024-09-11T08:37:51.270937+0000","last_became_peered":"2024-09-11T08:37:51.270937+0000","last_unstale":"2024-09-11T08:37:58.350038+0000","last_undegraded":"2024-09-11T08:37:58.350038+0000","last_fullsized":"2024-09-11T08:37:58.350038+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:01.796810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.482333+0000","last_change":"2024-09-11T08:37:49.018172+0000","last_active":"2024-09-11T08:37:58.482333+0000","last_peered":"2024-09-11T08:37:58.482333+0000","last_clean":"2024-09-11T08:37:58.482333+0000","last_became_active":"2024-09-11T08:37:49.017916+0000","last_became_peered":"2024-09-11T08:37:49.017916+0000","last_unstale":"2024-09-11T08:37:58.482333+0000","last_undegraded":"2024-09-11T08:37:58.482333+0000","last_fullsized":"2024-09-11T08:37:58.482333+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:01:32.572780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.522812+0000","last_change":"2024-09-11T08:37:53.049065+0000","last_active":"2024-09-11T08:37:59.522812+0000","last_peered":"2024-09-11T08:37:59.522812+0000","last_clean":"2024-09-11T08:37:59.522812+0000","last_became_active":"2024-09-11T08:37:53.048779+0000","last_became_peered":"2024-09-11T08:37:53.048779+0000","last_unstale":"2024-09-11T08:37:59.522812+0000","last_undegraded":"2024-09-11T08:37:59.522812+0000","last_fullsized":"2024-09-11T08:37:59.522812+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:12.708250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349149+0000","last_change":"2024-09-11T08:37:55.435290+0000","last_active":"2024-09-11T08:37:58.349149+0000","last_peered":"2024-09-11T08:37:58.349149+0000","last_clean":"2024-09-11T08:37:58.349149+0000","last_became_active":"2024-09-11T08:37:55.434752+0000","last_became_peered":"2024-09-11T08:37:55.434752+0000","last_unstale":"2024-09-11T08:37:58.349149+0000","last_undegraded":"2024-09-11T08:37:58.349149+0000","last_fullsized":"2024-09-11T08:37:58.349149+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:46:11.761668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.523310+0000","last_change":"2024-09-11T08:37:55.052787+0000","last_active":"2024-09-11T08:37:59.523310+0000","last_peered":"2024-09-11T08:37:59.523310+0000","last_clean":"2024-09-11T08:37:59.523310+0000","last_became_active":"2024-09-11T08:37:55.052485+0000","last_became_peered":"2024-09-11T08:37:55.052485+0000","last_unstale":"2024-09-11T08:37:59.523310+0000","last_undegraded":"2024-09-11T08:37:59.523310+0000","last_fullsized":"2024-09-11T08:37:59.523310+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:06.627631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.538562+0000","last_change":"2024-09-11T08:37:49.022999+0000","last_active":"2024-09-11T08:37:58.538562+0000","last_peered":"2024-09-11T08:37:58.538562+0000","last_clean":"2024-09-11T08:37:58.538562+0000","last_became_active":"2024-09-11T08:37:49.021669+0000","last_became_peered":"2024-09-11T08:37:49.021669+0000","last_unstale":"2024-09-11T08:37:58.538562+0000","last_undegraded":"2024-09-11T08:37:58.538562+0000","last_fullsized":"2024-09-11T08:37:58.538562+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:55.812373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.063039+0000","last_change":"2024-09-11T08:37:51.272436+0000","last_active":"2024-09-11T08:37:59.063039+0000","last_peered":"2024-09-11T08:37:59.063039+0000","last_clean":"2024-09-11T08:37:59.063039+0000","last_became_active":"2024-09-11T08:37:51.271729+0000","last_became_peered":"2024-09-11T08:37:51.271729+0000","last_unstale":"2024-09-11T08:37:59.063039+0000","last_undegraded":"2024-09-11T08:37:59.063039+0000","last_fullsized":"2024-09-11T08:37:59.063039+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:45:27.117726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"59'8","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:39:07.183561+0000","last_change":"2024-09-11T08:37:53.056346+0000","last_active":"2024-09-11T08:39:07.183561+0000","last_peered":"2024-09-11T08:39:07.183561+0000","last_clean":"2024-09-11T08:39:07.183561+0000","last_became_active":"2024-09-11T08:37:53.055198+0000","last_became_peered":"2024-09-11T08:37:53.055198+0000","last_unstale":"2024-09-11T08:39:07.183561+0000","last_undegraded":"2024-09-11T08:39:07.183561+0000","last_fullsized":"2024-09-11T08:39:07.183561+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:05:49.517104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455140+0000","last_change":"2024-09-11T08:37:55.049768+0000","last_active":"2024-09-11T08:37:58.455140+0000","last_peered":"2024-09-11T08:37:58.455140+0000","last_clean":"2024-09-11T08:37:58.455140+0000","last_became_active":"2024-09-11T08:37:55.049485+0000","last_became_peered":"2024-09-11T08:37:55.049485+0000","last_unstale":"2024-09-11T08:37:58.455140+0000","last_undegraded":"2024-09-11T08:37:58.455140+0000","last_fullsized":"2024-09-11T08:37:58.455140+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:51:56.447519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350233+0000","last_change":"2024-09-11T08:37:49.024005+0000","last_active":"2024-09-11T08:37:58.350233+0000","last_peered":"2024-09-11T08:37:58.350233+0000","last_clean":"2024-09-11T08:37:58.350233+0000","last_became_active":"2024-09-11T08:37:49.023743+0000","last_became_peered":"2024-09-11T08:37:49.023743+0000","last_unstale":"2024-09-11T08:37:58.350233+0000","last_undegraded":"2024-09-11T08:37:58.350233+0000","last_fullsized":"2024-09-11T08:37:58.350233+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:11:57.613960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.482293+0000","last_change":"2024-09-11T08:37:51.033795+0000","last_active":"2024-09-11T08:37:58.482293+0000","last_peered":"2024-09-11T08:37:58.482293+0000","last_clean":"2024-09-11T08:37:58.482293+0000","last_became_active":"2024-09-11T08:37:51.033530+0000","last_became_peered":"2024-09-11T08:37:51.033530+0000","last_unstale":"2024-09-11T08:37:58.482293+0000","last_undegraded":"2024-09-11T08:37:58.482293+0000","last_fullsized":"2024-09-11T08:37:58.482293+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:34.205495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:59.063009+0000","last_change":"2024-09-11T08:37:53.049681+0000","last_active":"2024-09-11T08:37:59.063009+0000","last_peered":"2024-09-11T08:37:59.063009+0000","last_clean":"2024-09-11T08:37:59.063009+0000","last_became_active":"2024-09-11T08:37:53.049391+0000","last_became_peered":"2024-09-11T08:37:53.049391+0000","last_unstale":"2024-09-11T08:37:59.063009+0000","last_undegraded":"2024-09-11T08:37:59.063009+0000","last_fullsized":"2024-09-11T08:37:59.063009+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:17.798497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349749+0000","last_change":"2024-09-11T08:37:55.434487+0000","last_active":"2024-09-11T08:37:58.349749+0000","last_peered":"2024-09-11T08:37:58.349749+0000","last_clean":"2024-09-11T08:37:58.349749+0000","last_became_active":"2024-09-11T08:37:55.434193+0000","last_became_peered":"2024-09-11T08:37:55.434193+0000","last_unstale":"2024-09-11T08:37:58.349749+0000","last_undegraded":"2024-09-11T08:37:58.349749+0000","last_fullsized":"2024-09-11T08:37:58.349749+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:33:31.392445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.482091+0000","last_change":"2024-09-11T08:37:49.019897+0000","last_active":"2024-09-11T08:37:58.482091+0000","last_peered":"2024-09-11T08:37:58.482091+0000","last_clean":"2024-09-11T08:37:58.482091+0000","last_became_active":"2024-09-11T08:37:49.019610+0000","last_became_peered":"2024-09-11T08:37:49.019610+0000","last_unstale":"2024-09-11T08:37:58.482091+0000","last_undegraded":"2024-09-11T08:37:58.482091+0000","last_fullsized":"2024-09-11T08:37:58.482091+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:38:28.312052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350204+0000","last_change":"2024-09-11T08:37:51.033099+0000","last_active":"2024-09-11T08:37:58.350204+0000","last_peered":"2024-09-11T08:37:58.350204+0000","last_clean":"2024-09-11T08:37:58.350204+0000","last_became_active":"2024-09-11T08:37:51.032802+0000","last_became_peered":"2024-09-11T08:37:51.032802+0000","last_unstale":"2024-09-11T08:37:58.350204+0000","last_undegraded":"2024-09-11T08:37:58.350204+0000","last_fullsized":"2024-09-11T08:37:58.350204+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:49:43.471481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.455432+0000","last_change":"2024-09-11T08:37:53.053004+0000","last_active":"2024-09-11T08:37:58.455432+0000","last_peered":"2024-09-11T08:37:58.455432+0000","last_clean":"2024-09-11T08:37:58.455432+0000","last_became_active":"2024-09-11T08:37:53.052732+0000","last_became_peered":"2024-09-11T08:37:53.052732+0000","last_unstale":"2024-09-11T08:37:58.455432+0000","last_undegraded":"2024-09-11T08:37:58.455432+0000","last_fullsized":"2024-09-11T08:37:58.455432+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:34:33.209197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350313+0000","last_change":"2024-09-11T08:37:55.434866+0000","last_active":"2024-09-11T08:37:58.350313+0000","last_peered":"2024-09-11T08:37:58.350313+0000","last_clean":"2024-09-11T08:37:58.350313+0000","last_became_active":"2024-09-11T08:37:55.434553+0000","last_became_peered":"2024-09-11T08:37:55.434553+0000","last_unstale":"2024-09-11T08:37:58.350313+0000","last_undegraded":"2024-09-11T08:37:58.350313+0000","last_fullsized":"2024-09-11T08:37:58.350313+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:54.033204+0000","last_clean_scrub_stamp":"2024-09-11T08:37:54.033204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:39:45.243465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350708+0000","last_change":"2024-09-11T08:37:49.022591+0000","last_active":"2024-09-11T08:37:58.350708+0000","last_peered":"2024-09-11T08:37:58.350708+0000","last_clean":"2024-09-11T08:37:58.350708+0000","last_became_active":"2024-09-11T08:37:49.022314+0000","last_became_peered":"2024-09-11T08:37:49.022314+0000","last_unstale":"2024-09-11T08:37:58.350708+0000","last_undegraded":"2024-09-11T08:37:58.350708+0000","last_fullsized":"2024-09-11T08:37:58.350708+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:48.000113+0000","last_clean_scrub_stamp":"2024-09-11T08:37:48.000113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:25.308957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.350626+0000","last_change":"2024-09-11T08:37:51.034941+0000","last_active":"2024-09-11T08:37:58.350626+0000","last_peered":"2024-09-11T08:37:58.350626+0000","last_clean":"2024-09-11T08:37:58.350626+0000","last_became_active":"2024-09-11T08:37:51.034571+0000","last_became_peered":"2024-09-11T08:37:51.034571+0000","last_unstale":"2024-09-11T08:37:58.350626+0000","last_undegraded":"2024-09-11T08:37:58.350626+0000","last_fullsized":"2024-09-11T08:37:58.350626+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:50.005859+0000","last_clean_scrub_stamp":"2024-09-11T08:37:50.005859+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:00:32.482653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:37:58.349778+0000","last_change":"2024-09-11T08:37:53.050703+0000","last_active":"2024-09-11T08:37:58.349778+0000","last_peered":"2024-09-11T08:37:58.349778+0000","last_clean":"2024-09-11T08:37:58.349778+0000","last_became_active":"2024-09-11T08:37:53.050429+0000","last_became_peered":"2024-09-11T08:37:53.050429+0000","last_unstale":"2024-09-11T08:37:58.349778+0000","last_undegraded":"2024-09-11T08:37:58.349778+0000","last_fullsized":"2024-09-11T08:37:58.349778+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:37:52.018050+0000","last_clean_scrub_stamp":"2024-09-11T08:37:52.018050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:51:57.937403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430229,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298536,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446616,"statfs":{"total":95995035648,"available":95689334784,"internally_reserved":0,"allocated":1208320,"data_stored":822690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.50900000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.54900000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":10.023,"5min":10.023,"15min":10.023},"last":0.48599999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.52200000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.375}]},{"osd":5,"last update":"Wed Sep 11 08:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.40000000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.59099999999999997}]}]},{"osd":6,"up_from":42,"seq":180388626457,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298508,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446644,"statfs":{"total":95995035648,"available":95689363456,"internally_reserved":0,"allocated":1183744,"data_stored":821798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.60299999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed Sep 11 08:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.48799999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.629}]},{"osd":4,"last update":"Wed Sep 11 08:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.64500000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58299999999999996}]},{"osd":7,"last update":"Wed Sep 11 08:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71399999999999997}]}]},{"osd":5,"up_from":36,"seq":154618822686,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":598016,"data_stored":232253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.502,"15min":0.502},"min":{"1min":0.38100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63900000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.504,"15min":0.499},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72199999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55200000000000005}]},{"osd":1,"last update":"Wed Sep 11 08:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.41499999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.501,"15min":0.504},"min":{"1min":0.38200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.63600000000000001,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.40799999999999997}]},{"osd":2,"last update":"Wed Sep 11 08:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52500000000000002,"15min":0.52600000000000002},"min":{"1min":0.32500000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66400000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52000000000000002,"15min":0.51900000000000002},"min":{"1min":0.373,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.38300000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65100000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.51200000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.379,"15min":0.375},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.40899999999999997},"min":{"1min":0.252,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.625}]},{"osd":6,"last update":"Wed Sep 11 08:38:14 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.60799999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:38:38 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.56299999999999994}]}]},{"osd":4,"up_from":30,"seq":128849018915,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297956,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447196,"statfs":{"total":95995035648,"available":95689928704,"internally_reserved":0,"allocated":614400,"data_stored":232113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64400000000000002,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.35499999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.65300000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.39800000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.30299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.38600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.623,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68400000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.55000000000000004,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.63800000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56899999999999995}]},{"osd":3,"last update":"Wed Sep 11 08:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.317,"5min":0.311,"15min":0.311},"max":{"1min":0.67400000000000004,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.39600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.71999999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.53100000000000003}]},{"osd":5,"last update":"Wed Sep 11 08:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.28100000000000003,"5min":0.253,"15min":0.253},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39700000000000002,"15min":0.39800000000000002},"min":{"1min":0.254,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63300000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.505}]},{"osd":6,"last update":"Wed Sep 11 08:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.67800000000000005}]},{"osd":7,"last update":"Wed Sep 11 08:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.63400000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182438,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297976,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447176,"statfs":{"total":95995035648,"available":95689908224,"internally_reserved":0,"allocated":634880,"data_stored":232826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.38300000000000001,"15min":0.39300000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.54500000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.379,"15min":0.374},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.53500000000000003,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.36799999999999999}]},{"osd":1,"last update":"Wed Sep 11 08:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.27300000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.58999999999999997,"5min":0.621,"15min":0.621},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40300000000000002,"15min":0.41099999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52500000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.41699999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.41299999999999998,"15min":0.42599999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56299999999999994,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40999999999999998,"15min":0.41599999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.55000000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.40000000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55100000000000005,"15min":0.55400000000000005},"min":{"1min":0.39200000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.71799999999999997,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.39500000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71399999999999997,"5min":0.874,"15min":0.874},"last":0.55700000000000005}]},{"osd":5,"last update":"Wed Sep 11 08:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73899999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.498},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78100000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.63700000000000001}]},{"osd":6,"last update":"Wed Sep 11 08:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:38:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.65100000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":598016,"data_stored":232116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:38:47 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.34200000000000003,"15min":0.33800000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.53800000000000003,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.307},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35699999999999998,"15min":0.34899999999999998},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.58599999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:38:47 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35499999999999998,"15min":0.34399999999999997},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.55100000000000005,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.83599999999999997,"15min":1.0389999999999999},"min":{"1min":0.248,"5min":0.246,"15min":0.246},"max":{"1min":0.629,"5min":15.186,"15min":15.186},"last":0.34999999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.41799999999999998,"15min":0.438},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.54400000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.42099999999999999,"15min":0.443},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.52100000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.38}]},{"osd":4,"last update":"Wed Sep 11 08:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74199999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40699999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76300000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.54400000000000004}]},{"osd":5,"last update":"Wed Sep 11 08:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.55000000000000004,"15min":0.55900000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71099999999999997,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82299999999999995,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.44600000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:38:42 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.60699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297988,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447164,"statfs":{"total":95995035648,"available":95689895936,"internally_reserved":0,"allocated":647168,"data_stored":233880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:38:23 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.32400000000000001,"15min":0.308},"min":{"1min":0.247,"5min":0.221,"15min":0.221},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.34699999999999998,"15min":0.32200000000000001},"min":{"1min":0.26400000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.55900000000000005,"5min":0.625,"15min":0.625},"last":0.51300000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:38:52 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34899999999999998,"15min":0.34300000000000003},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.49099999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.38,"15min":0.36799999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.52300000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:38:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.432,"15min":0.42499999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.38},"min":{"1min":0.30099999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.436}]},{"osd":4,"last update":"Wed Sep 11 08:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":3.746,"5min":3.746,"15min":3.746},"last":0.77100000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:38:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.75}]},{"osd":6,"last update":"Wed Sep 11 08:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.78300000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:38:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.81599999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298528,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446624,"statfs":{"total":95995035648,"available":95689342976,"internally_reserved":0,"allocated":1204224,"data_stored":822886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:38:27 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.313,"15min":0.29299999999999998},"min":{"1min":0.218,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.499,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.33100000000000002,"15min":0.307},"min":{"1min":0.249,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.26100000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35099999999999998,"15min":0.34300000000000003},"min":{"1min":0.23300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.379,"15min":0.371},"min":{"1min":0.245,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.32600000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:38:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52300000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.438,"15min":0.437},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.55500000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.35199999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:38:31 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55600000000000005}]},{"osd":5,"last update":"Wed Sep 11 08:38:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76000000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.626},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56899999999999995,"15min":0.57299999999999995},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85999999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.56899999999999995}]},{"osd":6,"last update":"Wed Sep 11 08:38:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.50600000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:38:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.54500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:39:17.493 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T08:39:17.493 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T08:39:17.493 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T08:39:17.493 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T08:39:17.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:17 smithi131 ceph-mon[37208]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:17.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:17 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:17 smithi131 ceph-mon[29446]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:17 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:17.737 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:17.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:17 smithi189 ceph-mon[32675]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:17.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:17 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:18.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:18 smithi189 ceph-mon[32675]: from='client.24647 v1:172.21.15.131:0/548498646' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:39:18.507 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:18 smithi131 ceph-mon[37208]: from='client.24647 v1:172.21.15.131:0/548498646' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:39:18.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:18 smithi131 ceph-mon[29446]: from='client.24647 v1:172.21.15.131:0/548498646' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:39:19.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:19 smithi131 ceph-mon[37208]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:19.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:19 smithi131 ceph-mon[29446]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:19.482 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-11T08:39:19.482 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T08:39:19.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:19 smithi189 ceph-mon[32675]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:19.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:39:19 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:39:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:39:20.066 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T08:39:20.066 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T08:39:20.067 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T08:39:20.081 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi131.front.sepia.ceph.com 2024-09-11T08:39:20.082 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 -- bash -c 'set -ex 2024-09-11T08:39:20.082 DEBUG:teuthology.orchestra.run.smithi131:> for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:39:20.082 DEBUG:teuthology.orchestra.run.smithi131:> do 2024-09-11T08:39:20.082 DEBUG:teuthology.orchestra.run.smithi131:> echo "rotating key for $f" 2024-09-11T08:39:20.082 DEBUG:teuthology.orchestra.run.smithi131:> K=$(ceph auth get-key $f) 2024-09-11T08:39:20.083 DEBUG:teuthology.orchestra.run.smithi131:> NK="$K" 2024-09-11T08:39:20.083 DEBUG:teuthology.orchestra.run.smithi131:> ceph orch daemon rotate-key $f 2024-09-11T08:39:20.083 DEBUG:teuthology.orchestra.run.smithi131:> while [ "$K" == "$NK" ]; do 2024-09-11T08:39:20.083 DEBUG:teuthology.orchestra.run.smithi131:> sleep 5 2024-09-11T08:39:20.083 DEBUG:teuthology.orchestra.run.smithi131:> NK=$(ceph auth get-key $f) 2024-09-11T08:39:20.083 DEBUG:teuthology.orchestra.run.smithi131:> done 2024-09-11T08:39:20.083 DEBUG:teuthology.orchestra.run.smithi131:> done 2024-09-11T08:39:20.083 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-11T08:39:20.352 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:39:20.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:20 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/945785667' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:39:20.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:20 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/945785667' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:39:20.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:20 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/945785667' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:39:21.096 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:39:21 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:39:21.464 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:21 smithi131 ceph-mon[37208]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:21.464 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:21 smithi131 ceph-mon[29446]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:21.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:21 smithi189 ceph-mon[32675]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:21.967 INFO:teuthology.orchestra.run.smithi131.stdout:rotating key for osd.0 2024-09-11T08:39:21.967 INFO:teuthology.orchestra.run.smithi131.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:39:21.967 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo 'rotating key for osd.0' 2024-09-11T08:39:21.968 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:39:22.140 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:21 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[70160]: ts=2024-09-11T08:39:21.942Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.00219818s 2024-09-11T08:39:22.194 INFO:teuthology.orchestra.run.smithi131.stderr:+ K=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:39:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:39:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:+ ceph orch daemon rotate-key osd.0 2024-09-11T08:39:22.451 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled to rotate-key osd.0 on host 'smithi131' 2024-09-11T08:39:22.451 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:22 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:22.451 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:22 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2968075015' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:22.452 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:22 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:22.452 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:22 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2968075015' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:22.461 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:39:22.462 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:39:22.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:22 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:22.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:22 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2968075015' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:22.741 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:22 smithi189 podman[63196]: 2024-09-11 08:39:22.409460861 +0000 UTC m=+0.014801687 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-11T08:39:23.143 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:22 smithi189 podman[63196]: 2024-09-11 08:39:22.816093471 +0000 UTC m=+0.421434339 container create 46bbf925bf21cee5be47aaff9c563e4adecfee1a0124c055fd3595c02d14b4c6 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.expose-services=, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, maintainer=Paul Cuzner , name=ubi8, io.buildah.version=1.32.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, distribution-scope=public, vendor=Red Hat, Inc., io.openshift.tags=base rhel8, architecture=x86_64, version=8.5, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=239.1651231664, description=Ceph Grafana Container, build-date=2022-04-29T11:48:11.055897, vcs-type=git, summary=Grafana Container configured for Ceph mgr/dashboard integration) 2024-09-11T08:39:23.143 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 podman[63196]: 2024-09-11 08:39:23.051591003 +0000 UTC m=+0.656931833 container init 46bbf925bf21cee5be47aaff9c563e4adecfee1a0124c055fd3595c02d14b4c6 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a, maintainer=Paul Cuzner , io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=239.1651231664, build-date=2022-04-29T11:48:11.055897, version=8.5, description=Ceph Grafana Container, io.buildah.version=1.32.0, io.openshift.expose-services=, vcs-type=git, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, io.openshift.tags=base rhel8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664) 2024-09-11T08:39:23.143 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 podman[63196]: 2024-09-11 08:39:23.05485524 +0000 UTC m=+0.660196075 container start 46bbf925bf21cee5be47aaff9c563e4adecfee1a0124c055fd3595c02d14b4c6 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, architecture=x86_64, version=8.5, com.redhat.component=ubi8-container, vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.tags=base rhel8, io.k8s.display-name=Red Hat Universal Base Image 8, distribution-scope=public, release=239.1651231664, description=Ceph Grafana Container, io.openshift.expose-services=, io.buildah.version=1.32.0, build-date=2022-04-29T11:48:11.055897, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, maintainer=Paul Cuzner , io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=ubi8, vendor=Red Hat, Inc.) 2024-09-11T08:39:23.143 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 bash[63196]: 46bbf925bf21cee5be47aaff9c563e4adecfee1a0124c055fd3595c02d14b4c6 2024-09-11T08:39:23.393 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 systemd[1]: Started Ceph grafana.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:39:23.393 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=settings t=2024-09-11T08:39:23.216044895Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-11T08:39:23.393 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=settings t=2024-09-11T08:39:23.216230943Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-11T08:39:23.394 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=settings t=2024-09-11T08:39:23.216255706Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-11T08:39:23.394 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=settings t=2024-09-11T08:39:23.216266682Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-11T08:39:23.394 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=settings t=2024-09-11T08:39:23.216275445Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-11T08:39:23.394 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=settings t=2024-09-11T08:39:23.216285207Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-11T08:39:23.394 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=settings t=2024-09-11T08:39:23.216293695Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-11T08:39:23.394 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=settings t=2024-09-11T08:39:23.216302356Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-11T08:39:23.394 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=settings t=2024-09-11T08:39:23.216310716Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-11T08:39:23.394 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=settings t=2024-09-11T08:39:23.216319656Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-11T08:39:23.394 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=settings t=2024-09-11T08:39:23.216327522Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-11T08:39:23.394 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=settings t=2024-09-11T08:39:23.216335187Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-11T08:39:23.394 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=settings t=2024-09-11T08:39:23.21634478Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-11T08:39:23.394 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=settings t=2024-09-11T08:39:23.216356465Z level=info msg="App mode production" 2024-09-11T08:39:23.395 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=sqlstore t=2024-09-11T08:39:23.216417182Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-11T08:39:23.395 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=sqlstore t=2024-09-11T08:39:23.216439963Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-11T08:39:23.395 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.217715436Z level=info msg="Starting DB migrations" 2024-09-11T08:39:23.395 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.217828446Z level=info msg="Executing migration" id="create migration_log table" 2024-09-11T08:39:23.395 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.218853347Z level=info msg="Executing migration" id="create user table" 2024-09-11T08:39:23.395 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.219302447Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-11T08:39:23.395 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.221119005Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-11T08:39:23.395 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.22479184Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-11T08:39:23.395 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.225263364Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-11T08:39:23.395 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.225650816Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-11T08:39:23.395 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.226432494Z level=info msg="Executing migration" id="create user table v2" 2024-09-11T08:39:23.395 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.227560081Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-11T08:39:23.396 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.229462874Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-11T08:39:23.396 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.233240705Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-11T08:39:23.396 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.233602654Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-11T08:39:23.396 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.235465811Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-11T08:39:23.396 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.235915122Z level=info msg="Executing migration" id="Update user table charset" 2024-09-11T08:39:23.396 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.237366347Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-11T08:39:23.396 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.239275447Z level=info msg="Executing migration" id="Add missing user data" 2024-09-11T08:39:23.396 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.241149271Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-11T08:39:23.396 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.243054929Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-11T08:39:23.396 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.244940038Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-11T08:39:23.396 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.246981097Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-11T08:39:23.396 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.248860719Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-11T08:39:23.398 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.249297225Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-11T08:39:23.398 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.24976541Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-11T08:39:23.398 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.250205819Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-11T08:39:23.398 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.250618873Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-11T08:39:23.398 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.251044444Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-11T08:39:23.398 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.251365234Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-11T08:39:23.398 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.251745352Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-11T08:39:23.398 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.252178613Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-11T08:39:23.398 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.252588697Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-11T08:39:23.398 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.253007297Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-11T08:39:23.398 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.25366573Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-11T08:39:23.398 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.25409766Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-11T08:39:23.399 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.254522152Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-11T08:39:23.399 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.254912977Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-11T08:39:23.399 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.255328099Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-11T08:39:23.399 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.255744499Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-11T08:39:23.399 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.256126732Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-11T08:39:23.399 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.256472505Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-11T08:39:23.399 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.256804321Z level=info msg="Executing migration" id="create star table" 2024-09-11T08:39:23.399 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.257202002Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-11T08:39:23.399 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.257637089Z level=info msg="Executing migration" id="create org table v1" 2024-09-11T08:39:23.399 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.258059847Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-11T08:39:23.399 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.258480783Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-11T08:39:23.399 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.258893393Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-11T08:39:23.399 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.259316256Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-11T08:39:23.399 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.259716517Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-11T08:39:23.400 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.260144161Z level=info msg="Executing migration" id="Update org table charset" 2024-09-11T08:39:23.400 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.260925959Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-11T08:39:23.400 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.261783619Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-11T08:39:23.400 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.262492769Z level=info msg="Executing migration" id="create dashboard table" 2024-09-11T08:39:23.400 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.263268429Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-11T08:39:23.400 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.264054011Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-11T08:39:23.400 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.264904866Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-11T08:39:23.400 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.265770777Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-11T08:39:23.400 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.268358401Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-11T08:39:23.400 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.270937796Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-11T08:39:23.400 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.273706962Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-11T08:39:23.400 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.276425027Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-11T08:39:23.400 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.27913159Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-11T08:39:23.400 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.281872288Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-11T08:39:23.401 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.284595687Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-11T08:39:23.401 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.287250492Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-11T08:39:23.401 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.289999394Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-11T08:39:23.401 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.292903263Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-11T08:39:23.401 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.295657097Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-11T08:39:23.401 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.298262335Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-11T08:39:23.401 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.301140541Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-11T08:39:23.401 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.303809923Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-11T08:39:23.401 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.306316587Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-11T08:39:23.401 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.309113827Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-11T08:39:23.401 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.311627637Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-11T08:39:23.401 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.314319593Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-11T08:39:23.401 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.316522357Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-11T08:39:23.401 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.317116499Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-11T08:39:23.402 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.319904839Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-11T08:39:23.402 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.322516745Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-11T08:39:23.402 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.324991864Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-11T08:39:23.402 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.32759882Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-11T08:39:23.402 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.330071677Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-11T08:39:23.402 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.332546914Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-11T08:39:23.402 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.33489817Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-11T08:39:23.402 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.337311171Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-11T08:39:23.402 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.33976509Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-11T08:39:23.402 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.342251746Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-11T08:39:23.402 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.344588185Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-11T08:39:23.402 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.347170493Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-11T08:39:23.402 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.349499107Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-11T08:39:23.402 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.351764977Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-11T08:39:23.403 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.354282083Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-11T08:39:23.403 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.356778343Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-11T08:39:23.403 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.35754495Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-11T08:39:23.403 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.358283099Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-11T08:39:23.403 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.358725322Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-11T08:39:23.403 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.359262138Z level=info msg="Executing migration" id="create data_source table" 2024-09-11T08:39:23.403 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.360122816Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-11T08:39:23.403 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.360744841Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-11T08:39:23.403 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.361175987Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-11T08:39:23.403 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.361929315Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-11T08:39:23.403 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.362797124Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-11T08:39:23.403 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.363990199Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-11T08:39:23.403 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.364889403Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-11T08:39:23.403 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.365531529Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-11T08:39:23.404 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.366276547Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-11T08:39:23.404 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.367027303Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-11T08:39:23.404 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.36779138Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-11T08:39:23.404 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.368507391Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-11T08:39:23.404 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.369265211Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-11T08:39:23.404 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.369636147Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-11T08:39:23.404 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.370323621Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-11T08:39:23.404 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.371061276Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-11T08:39:23.404 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.371790222Z level=info msg="Executing migration" id="Add uid column" 2024-09-11T08:39:23.404 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.372541489Z level=info msg="Executing migration" id="Update uid value" 2024-09-11T08:39:23.404 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.373301741Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-11T08:39:23.404 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.373937659Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-11T08:39:23.404 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.374406361Z level=info msg="Executing migration" id="create api_key table" 2024-09-11T08:39:23.405 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.375091638Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-11T08:39:23.405 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.37584058Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-11T08:39:23.405 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.37656938Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-11T08:39:23.405 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.377350137Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-11T08:39:23.405 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.378037369Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-11T08:39:23.405 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.378908763Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-11T08:39:23.405 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.379491447Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-11T08:39:23.405 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.380585086Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-11T08:39:23.405 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.381238325Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-11T08:39:23.405 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.381686907Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-11T08:39:23.405 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.382135814Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-11T08:39:23.405 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.382544401Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-11T08:39:23.405 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.382851583Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-11T08:39:23.405 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.383233839Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-11T08:39:23.405 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.383506226Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-11T08:39:23.406 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.384045193Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-11T08:39:23.406 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.384612232Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-11T08:39:23.406 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.384898569Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-11T08:39:23.406 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.385489889Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-11T08:39:23.406 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.386036197Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-11T08:39:23.406 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.38645731Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-11T08:39:23.406 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.386818643Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-11T08:39:23.406 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.387243853Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-11T08:39:23.406 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.387662801Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-11T08:39:23.406 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.388076404Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-11T08:39:23.406 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.38850781Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-11T08:39:23.406 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.388767641Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-11T08:39:23.406 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.389085469Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-11T08:39:23.406 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.389607723Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-11T08:39:23.407 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.390195032Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-11T08:39:23.407 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.390505098Z level=info msg="Executing migration" id="create quota table v1" 2024-09-11T08:39:23.407 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.390890882Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-11T08:39:23.407 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.391334795Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-11T08:39:23.407 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.391631815Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-11T08:39:23.407 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.392037766Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-11T08:39:23.407 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.392473605Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-11T08:39:23.407 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.393100567Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-11T08:39:23.407 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.393392388Z level=info msg="Executing migration" id="create session table" 2024-09-11T08:39:23.407 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.393838066Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-11T08:39:23.407 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.394168021Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-11T08:39:23.407 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.394459927Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-11T08:39:23.407 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.394915706Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-11T08:39:23.407 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.395307722Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-11T08:39:23.408 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.395576977Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-11T08:39:23.408 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.395851988Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-11T08:39:23.408 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.396156844Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-11T08:39:23.644 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-mon[32675]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:23.644 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-mon[32675]: from='client.14748 v1:172.21.15.131:0/1463393045' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:39:23.644 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-mon[32675]: Schedule rotate-key daemon osd.0 2024-09-11T08:39:23.644 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.645 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.645 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.645 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.645 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.645 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.648 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.39816426Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-11T08:39:23.648 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.398582958Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-11T08:39:23.648 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.400952691Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-11T08:39:23.648 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.401557596Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-11T08:39:23.648 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.403660259Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-11T08:39:23.648 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.40427241Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-11T08:39:23.648 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.40489033Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-11T08:39:23.648 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.407592762Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-11T08:39:23.648 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.410166036Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-11T08:39:23.648 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.412645452Z level=info msg="Executing migration" id="create alert table v1" 2024-09-11T08:39:23.648 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.413122086Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-11T08:39:23.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.415909625Z level=info msg="Executing migration" id="add index alert state" 2024-09-11T08:39:23.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.418436748Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-11T08:39:23.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.421003745Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-11T08:39:23.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.421409162Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-11T08:39:23.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.423983354Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-11T08:39:23.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.426750052Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-11T08:39:23.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.42924572Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-11T08:39:23.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.429652826Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-11T08:39:23.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.432481915Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-11T08:39:23.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.433083329Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-11T08:39:23.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.433935889Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-11T08:39:23.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.434338186Z level=info msg="Executing migration" id="Add column is_default" 2024-09-11T08:39:23.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.435199933Z level=info msg="Executing migration" id="Add column frequency" 2024-09-11T08:39:23.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.43607592Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-11T08:39:23.650 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.436695679Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-11T08:39:23.650 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.437656141Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-11T08:39:23.650 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.438122189Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-11T08:39:23.650 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.438892551Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-11T08:39:23.650 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.439764976Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-11T08:39:23.650 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.440711053Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-11T08:39:23.650 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.441280863Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-11T08:39:23.650 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.442126139Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-11T08:39:23.650 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.442582609Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-11T08:39:23.650 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.4430386Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-11T08:39:23.650 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.443792777Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-11T08:39:23.650 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.444523619Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-11T08:39:23.650 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.444862996Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-11T08:39:23.651 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.445744841Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-11T08:39:23.651 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.446589156Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-11T08:39:23.651 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.447467846Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-11T08:39:23.651 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.44830169Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-11T08:39:23.651 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.449276125Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-11T08:39:23.651 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.45001154Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-11T08:39:23.651 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.450963073Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-11T08:39:23.651 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.45177904Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-11T08:39:23.651 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.452617556Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-11T08:39:23.651 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.453441962Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-11T08:39:23.651 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.453915992Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-11T08:39:23.651 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.454761827Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-11T08:39:23.651 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.455580968Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-11T08:39:23.651 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.455890105Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-11T08:39:23.652 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.4578996Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-11T08:39:23.652 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.466311997Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-11T08:39:23.652 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.467310426Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-11T08:39:23.652 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.468099332Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-11T08:39:23.652 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.468897321Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-11T08:39:23.652 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.469689246Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-11T08:39:23.652 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.471330639Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-11T08:39:23.652 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.471735372Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-11T08:39:23.652 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.472591883Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-11T08:39:23.652 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.47347229Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-11T08:39:23.652 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.474312349Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-11T08:39:23.652 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.479108939Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-11T08:39:23.652 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.481640567Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-11T08:39:23.653 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.482324536Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-11T08:39:23.653 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.483122142Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-11T08:39:23.653 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.483927838Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-11T08:39:23.653 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.484814046Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-11T08:39:23.653 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.485530737Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-11T08:39:23.653 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.486472203Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-11T08:39:23.653 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.487212663Z level=info msg="Executing migration" id="Move region to single row" 2024-09-11T08:39:23.653 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.487999663Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-11T08:39:23.653 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.488795118Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-11T08:39:23.653 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.48978848Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-11T08:39:23.653 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.490492809Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-11T08:39:23.653 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.491254289Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-11T08:39:23.653 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.491687726Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-11T08:39:23.653 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.492131214Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-11T08:39:23.654 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.492996999Z level=info msg="Executing migration" id="create test_data table" 2024-09-11T08:39:23.654 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.493962796Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-11T08:39:23.654 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.494844519Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-11T08:39:23.654 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.495579708Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-11T08:39:23.654 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.496460934Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-11T08:39:23.654 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.497232635Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-11T08:39:23.654 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.498154935Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-11T08:39:23.654 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.498917595Z level=info msg="Executing migration" id="create team table" 2024-09-11T08:39:23.654 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.499938417Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-11T08:39:23.654 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.500655918Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-11T08:39:23.654 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.501530658Z level=info msg="Executing migration" id="create team member table" 2024-09-11T08:39:23.654 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.502311545Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-11T08:39:23.654 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.503145545Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-11T08:39:23.654 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.503998269Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-11T08:39:23.654 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.504840603Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-11T08:39:23.655 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.505690402Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-11T08:39:23.655 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.508544733Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-11T08:39:23.655 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.509318151Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-11T08:39:23.655 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.512064248Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-11T08:39:23.655 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.514908468Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-11T08:39:23.655 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.517604293Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-11T08:39:23.655 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.518192479Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-11T08:39:23.655 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.52222005Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-11T08:39:23.655 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.52596888Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-11T08:39:23.655 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.526530136Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-11T08:39:23.655 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.526942549Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-11T08:39:23.655 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.527336326Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-11T08:39:23.655 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.527619561Z level=info msg="Executing migration" id="create tag table" 2024-09-11T08:39:23.655 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.528008982Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-11T08:39:23.655 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.528424914Z level=info msg="Executing migration" id="create login attempt table" 2024-09-11T08:39:23.655 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.528799447Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-11T08:39:23.656 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.52924115Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-11T08:39:23.656 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.529616803Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-11T08:39:23.656 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.531521639Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-11T08:39:23.656 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.531913015Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-11T08:39:23.656 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.532332514Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-11T08:39:23.656 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.532621308Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-11T08:39:23.656 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.532980444Z level=info msg="Executing migration" id="create user auth table" 2024-09-11T08:39:23.656 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.533352849Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-11T08:39:23.656 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.53375869Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-11T08:39:23.656 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.534050674Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-11T08:39:23.656 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.534767203Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-11T08:39:23.656 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.535568815Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-11T08:39:23.656 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.536353322Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-11T08:39:23.657 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.537111796Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-11T08:39:23.657 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.537529379Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-11T08:39:23.657 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.538325559Z level=info msg="Executing migration" id="create server_lock table" 2024-09-11T08:39:23.657 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.538711537Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-11T08:39:23.657 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.539144569Z level=info msg="Executing migration" id="create user auth token table" 2024-09-11T08:39:23.657 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.53956411Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-11T08:39:23.657 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.543558565Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-11T08:39:23.657 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.544086266Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-11T08:39:23.657 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.544514305Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-11T08:39:23.657 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.545287378Z level=info msg="Executing migration" id="create cache_data table" 2024-09-11T08:39:23.657 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.545726689Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-11T08:39:23.657 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.546153706Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-11T08:39:23.657 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.546572304Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-11T08:39:23.657 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.550499659Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-11T08:39:23.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.550878781Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-11T08:39:23.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.55119253Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-11T08:39:23.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.551654798Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-11T08:39:23.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.55543399Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-11T08:39:23.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.55592735Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-11T08:39:23.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.55623239Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-11T08:39:23.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.556634324Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-11T08:39:23.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.557068805Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-11T08:39:23.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.55751224Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-11T08:39:23.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.55797986Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-11T08:39:23.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.558819838Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-11T08:39:23.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.559306461Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-11T08:39:23.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.559594644Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-11T08:39:23.658 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.560248114Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-11T08:39:23.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.560660202Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-11T08:39:23.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.561131772Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-11T08:39:23.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.561432331Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-11T08:39:23.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.561889314Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-11T08:39:23.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.562346063Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-11T08:39:23.659 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.562752001Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-11T08:39:23.660 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.563228845Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-11T08:39:23.660 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.564055601Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-11T08:39:23.660 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.56450232Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-11T08:39:23.660 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.56488586Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-11T08:39:23.660 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.568352957Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-11T08:39:23.660 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.571894331Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-11T08:39:23.660 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.57236543Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-11T08:39:23.660 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.572756115Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-11T08:39:23.660 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.573595616Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-11T08:39:23.660 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.576246345Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-11T08:39:23.660 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.580381134Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-11T08:39:23.660 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.584195955Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-11T08:39:23.660 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.5879585Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-11T08:39:23.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.591813693Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-11T08:39:23.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.595596073Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-11T08:39:23.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.599309734Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-11T08:39:23.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.603307302Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-11T08:39:23.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.607228221Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-11T08:39:23.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.611019069Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-11T08:39:23.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.612932377Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-11T08:39:23.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.614902656Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-11T08:39:23.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.616638323Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-11T08:39:23.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.618471159Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-11T08:39:23.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.620287185Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-11T08:39:23.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.620627332Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-11T08:39:23.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.622456674Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-11T08:39:23.661 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.624434864Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-11T08:39:23.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.624961978Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-11T08:39:23.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.626188962Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-11T08:39:23.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.628118894Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-11T08:39:23.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.629879931Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-11T08:39:23.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.631852856Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-11T08:39:23.662 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.633541299Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-11T08:39:23.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.635566447Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-11T08:39:23.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.637257981Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-11T08:39:23.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.639147106Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-11T08:39:23.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.640972593Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-11T08:39:23.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.64290888Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-11T08:39:23.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.644786869Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-11T08:39:23.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.646690701Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-11T08:39:23.663 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.648569501Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-11T08:39:23.745 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[29446]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:23.746 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[29446]: from='client.14748 v1:172.21.15.131:0/1463393045' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:39:23.746 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[29446]: Schedule rotate-key daemon osd.0 2024-09-11T08:39:23.746 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.746 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.746 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.747 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.747 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.747 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.747 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[37208]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:23.748 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[37208]: from='client.14748 v1:172.21.15.131:0/1463393045' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:39:23.748 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[37208]: Schedule rotate-key daemon osd.0 2024-09-11T08:39:23.748 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.748 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.748 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.748 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.749 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.749 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:23 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:23.895 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.650432918Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-11T08:39:23.895 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.652395244Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-11T08:39:23.895 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.654242912Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-11T08:39:23.895 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.656200407Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-11T08:39:23.895 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.657980775Z level=info msg="Executing migration" id="create alert_image table" 2024-09-11T08:39:23.895 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.660025355Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-11T08:39:23.895 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.661818947Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-11T08:39:23.895 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.663643042Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-11T08:39:23.896 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.665574311Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-11T08:39:23.896 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.666303247Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T08:39:23.896 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.666382266Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T08:39:23.896 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.668383144Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-11T08:39:23.896 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.67003133Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-11T08:39:23.896 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.672055427Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-11T08:39:23.896 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.672352541Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-11T08:39:23.896 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.67246678Z level=info msg="alerts found to migrate" alerts=0 2024-09-11T08:39:23.896 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.673846888Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-11T08:39:23.896 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.675790091Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-11T08:39:23.897 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.679733812Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-11T08:39:23.897 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.683638161Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-11T08:39:23.897 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.687422041Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-11T08:39:23.897 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.691427692Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-11T08:39:23.898 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.695339377Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-11T08:39:23.898 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.699122419Z level=info msg="Executing migration" id="create data_keys table" 2024-09-11T08:39:23.898 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.702998759Z level=info msg="Executing migration" id="create secrets table" 2024-09-11T08:39:23.898 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.706903658Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-11T08:39:23.898 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.711637488Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-11T08:39:23.898 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.716024299Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-11T08:39:23.898 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.719975425Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-11T08:39:23.898 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.724268416Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-11T08:39:23.898 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.72851767Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-11T08:39:23.899 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.732588592Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-11T08:39:23.899 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.73633464Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-11T08:39:23.899 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.736680512Z level=info msg="Executing migration" id="create permission table" 2024-09-11T08:39:23.899 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.740948423Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-11T08:39:23.899 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.744919382Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-11T08:39:23.899 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.746734458Z level=info msg="Executing migration" id="create role table" 2024-09-11T08:39:23.899 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.748587265Z level=info msg="Executing migration" id="add column display_name" 2024-09-11T08:39:23.899 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.750527246Z level=info msg="Executing migration" id="add column group_name" 2024-09-11T08:39:23.899 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.752296394Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-11T08:39:23.899 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.754530877Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-11T08:39:23.899 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.756668542Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-11T08:39:23.899 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.758128122Z level=info msg="Executing migration" id="create team role table" 2024-09-11T08:39:23.899 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.760968846Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-11T08:39:23.899 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.763784871Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-11T08:39:23.900 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.766651211Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-11T08:39:23.900 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.769185625Z level=info msg="Executing migration" id="create user role table" 2024-09-11T08:39:23.900 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.771887188Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-11T08:39:23.900 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.774697839Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-11T08:39:23.900 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.777492285Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-11T08:39:23.900 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.780324771Z level=info msg="Executing migration" id="create builtin role table" 2024-09-11T08:39:23.900 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.783290704Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-11T08:39:23.900 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.78603741Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-11T08:39:23.900 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.788768153Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-11T08:39:23.900 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.791467605Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-11T08:39:23.900 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.794140803Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-11T08:39:23.900 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.797032653Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-11T08:39:23.900 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.7997441Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-11T08:39:23.901 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.802555446Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-11T08:39:23.901 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.805246471Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-11T08:39:23.901 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.807979455Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-11T08:39:23.901 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.810505276Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-11T08:39:23.901 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.813300232Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-11T08:39:23.901 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.815970553Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-11T08:39:23.901 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.81873762Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-11T08:39:23.901 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.82145523Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-11T08:39:23.901 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.824074237Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-11T08:39:23.901 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.826873752Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-11T08:39:23.901 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.829522478Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-11T08:39:23.901 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.832260326Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-11T08:39:23.901 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.834764774Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-11T08:39:23.901 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.837614983Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-11T08:39:23.902 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.84012732Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-11T08:39:23.902 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.843073161Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-11T08:39:23.905 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.845694691Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-11T08:39:23.905 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.846531071Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-11T08:39:23.905 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.847407766Z level=info msg="Executing migration" id="add correlation config column" 2024-09-11T08:39:23.905 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.848587761Z level=info msg="Executing migration" id="create entity_events table" 2024-09-11T08:39:23.905 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.849320775Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-11T08:39:23.905 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.85032138Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:39:23.905 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.850534445Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:39:23.905 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.851032278Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:39:23.906 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.851121672Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:39:23.906 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.851803194Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-11T08:39:23.906 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.852793585Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-11T08:39:23.906 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.853516352Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:39:23.906 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.854353946Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:39:23.906 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.855244107Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-11T08:39:23.906 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.856071888Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T08:39:23.906 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.856918191Z level=info msg="Executing migration" id="Drop public config table" 2024-09-11T08:39:23.907 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.857884173Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-11T08:39:23.907 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.858562862Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-11T08:39:23.907 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.859421254Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T08:39:23.907 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.860408975Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-11T08:39:23.907 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.861118772Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-11T08:39:23.907 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.864264558Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-11T08:39:23.907 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.865299772Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-11T08:39:23.907 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.866333407Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-11T08:39:23.908 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.867207944Z level=info msg="Executing migration" id="add share column" 2024-09-11T08:39:23.908 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.868219485Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-11T08:39:23.908 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.869060802Z level=info msg="Executing migration" id="create file table" 2024-09-11T08:39:23.908 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.869938188Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-11T08:39:23.908 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.870820551Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-11T08:39:23.908 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.871627465Z level=info msg="Executing migration" id="create file_meta table" 2024-09-11T08:39:23.908 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.872480517Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-11T08:39:23.908 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.873473052Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-11T08:39:23.908 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.874232148Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-11T08:39:23.908 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.875179841Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-11T08:39:23.908 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.875942069Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-11T08:39:23.908 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.876797701Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-11T08:39:23.908 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.87787787Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-11T08:39:23.908 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.878826932Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-11T08:39:23.909 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.881605975Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-11T08:39:23.909 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.884408026Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-11T08:39:23.909 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.887165681Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-11T08:39:23.909 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.889909034Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-11T08:39:23.909 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.892689864Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-11T08:39:23.909 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.895680186Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-11T08:39:23.909 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.898231782Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-11T08:39:24.113 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:23 smithi131 systemd[1]: Starting Ceph node-exporter.a for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:39:24.218 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.90806806Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-11T08:39:24.219 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.910748936Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-11T08:39:24.219 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.913630813Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-11T08:39:24.219 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.917394059Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-11T08:39:24.219 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.920240959Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-11T08:39:24.219 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.923126868Z level=info msg="Executing migration" id="create folder table" 2024-09-11T08:39:24.219 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.925839589Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-11T08:39:24.219 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.928633025Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-11T08:39:24.219 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.931289789Z level=info msg="Executing migration" id="Update folder title length" 2024-09-11T08:39:24.219 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.93397523Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-11T08:39:24.219 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=migrator t=2024-09-11T08:39:23.936782399Z level=info msg="migrations completed" performed=485 skipped=0 duration=718.989531ms 2024-09-11T08:39:24.219 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=sqlstore t=2024-09-11T08:39:23.937202547Z level=info msg="Created default organization" 2024-09-11T08:39:24.219 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:23 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=plugin.loader t=2024-09-11T08:39:23.997301337Z level=info msg="Plugin registered" pluginID=input 2024-09-11T08:39:24.219 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=plugin.loader t=2024-09-11T08:39:24.025507446Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-11T08:39:24.220 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=plugin.loader t=2024-09-11T08:39:24.025532122Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-11T08:39:24.220 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=secrets t=2024-09-11T08:39:24.025569385Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-11T08:39:24.220 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=query_data t=2024-09-11T08:39:24.027295826Z level=info msg="Query Service initialization" 2024-09-11T08:39:24.220 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=live.push_http t=2024-09-11T08:39:24.029646493Z level=info msg="Live Push Gateway initialization" 2024-09-11T08:39:24.391 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:24 smithi131 bash[71048]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-11T08:39:24.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-mon[32675]: Deploying daemon node-exporter.a on smithi131 2024-09-11T08:39:24.787 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:24 smithi131 ceph-mon[37208]: Deploying daemon node-exporter.a on smithi131 2024-09-11T08:39:24.788 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:24 smithi131 ceph-mon[29446]: Deploying daemon node-exporter.a on smithi131 2024-09-11T08:39:24.788 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:24 smithi131 bash[71048]: Getting image source signatures 2024-09-11T08:39:24.788 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:24 smithi131 bash[71048]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-11T08:39:24.788 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:24 smithi131 bash[71048]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T08:39:24.788 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:24 smithi131 bash[71048]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-11T08:39:25.241 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=infra.usagestats.collector t=2024-09-11T08:39:24.782683487Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-11T08:39:25.241 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=provisioning.datasources t=2024-09-11T08:39:24.782993499Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-11T08:39:25.241 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=provisioning.datasources t=2024-09-11T08:39:24.816459467Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-11T08:39:25.241 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=provisioning.datasources t=2024-09-11T08:39:24.848451611Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-11T08:39:25.242 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=provisioning.alerting t=2024-09-11T08:39:24.869644184Z level=info msg="starting to provision alerting" 2024-09-11T08:39:25.242 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=provisioning.alerting t=2024-09-11T08:39:24.869681916Z level=info msg="finished to provision alerting" 2024-09-11T08:39:25.242 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=grafanaStorageLogger t=2024-09-11T08:39:24.869870861Z level=info msg="storage starting" 2024-09-11T08:39:25.242 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=ngalert.state.manager t=2024-09-11T08:39:24.869932208Z level=info msg="Warming state cache for startup" 2024-09-11T08:39:25.242 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=http.server t=2024-09-11T08:39:24.872274756Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-11T08:39:25.243 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=sqlstore.transactions t=2024-09-11T08:39:24.880107008Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:39:25.243 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=ngalert.state.manager t=2024-09-11T08:39:24.882118588Z level=info msg="State cache has been initialized" states=0 duration=12.185806ms 2024-09-11T08:39:25.243 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=ticker t=2024-09-11T08:39:24.882166467Z level=info msg=starting first_tick=2024-09-11T08:39:30Z 2024-09-11T08:39:25.243 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=ngalert.multiorg.alertmanager t=2024-09-11T08:39:24.882182045Z level=info msg="starting MultiOrg Alertmanager" 2024-09-11T08:39:25.243 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=sqlstore.transactions t=2024-09-11T08:39:24.93398411Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:39:25.243 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:39:24 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a[63211]: logger=sqlstore.transactions t=2024-09-11T08:39:24.944887438Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-11T08:39:25.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:25 smithi131 ceph-mon[29446]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:25.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:25 smithi131 ceph-mon[37208]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:25.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:25 smithi189 ceph-mon[32675]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:26.390 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 bash[71048]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T08:39:26.391 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 bash[71048]: Writing manifest to image destination 2024-09-11T08:39:26.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:26 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-09-11T08:39:26.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:26 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T08:39:26.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:26 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-09-11T08:39:26.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:26 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-11T08:39:26.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:26 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T08:39:26.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:26 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-09-11T08:39:26.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:26 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-11T08:39:26.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:26 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-09-11T08:39:26.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:26 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:26.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:26 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:26.770 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 podman[71048]: 2024-09-11 08:39:26.52097748 +0000 UTC m=+2.497834372 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T08:39:26.770 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 podman[71048]: 2024-09-11 08:39:26.644419668 +0000 UTC m=+2.621276568 container create 82e0910de1da02fb6403d51d8142224b44f4329c3f18ceb62edba2e63aa546cf (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:26.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-09-11T08:39:26.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T08:39:26.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-09-11T08:39:26.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-11T08:39:26.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T08:39:26.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-09-11T08:39:26.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-11T08:39:26.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-09-11T08:39:26.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:26.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:26.772 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-09-11T08:39:26.772 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T08:39:26.772 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-09-11T08:39:26.772 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-11T08:39:26.772 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T08:39:26.772 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-09-11T08:39:26.772 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-11T08:39:26.772 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-09-11T08:39:26.772 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:26.772 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:27.391 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 podman[71048]: 2024-09-11 08:39:26.923486362 +0000 UTC m=+2.900343263 container init 82e0910de1da02fb6403d51d8142224b44f4329c3f18ceb62edba2e63aa546cf (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:27.391 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 podman[71048]: 2024-09-11 08:39:26.926406596 +0000 UTC m=+2.903263496 container start 82e0910de1da02fb6403d51d8142224b44f4329c3f18ceb62edba2e63aa546cf (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:27.391 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.930Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T08:39:27.391 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.930Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T08:39:27.391 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T08:39:27.391 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T08:39:27.391 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T08:39:27.392 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T08:39:27.392 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T08:39:27.392 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T08:39:27.392 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T08:39:27.392 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T08:39:27.392 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T08:39:27.392 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T08:39:27.392 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T08:39:27.392 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T08:39:27.393 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T08:39:27.393 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T08:39:27.393 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T08:39:27.393 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T08:39:27.393 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T08:39:27.393 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T08:39:27.393 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T08:39:27.393 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T08:39:27.393 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T08:39:27.393 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T08:39:27.394 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T08:39:27.394 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T08:39:27.394 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T08:39:27.394 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.931Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T08:39:27.394 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T08:39:27.394 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T08:39:27.394 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T08:39:27.394 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T08:39:27.394 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T08:39:27.394 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T08:39:27.394 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T08:39:27.394 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T08:39:27.395 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T08:39:27.395 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T08:39:27.395 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T08:39:27.395 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T08:39:27.395 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T08:39:27.395 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T08:39:27.395 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T08:39:27.395 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T08:39:27.395 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T08:39:27.395 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T08:39:27.395 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T08:39:27.395 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T08:39:27.395 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T08:39:27.395 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T08:39:27.396 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T08:39:27.396 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T08:39:27.396 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a[71123]: ts=2024-09-11T08:39:26.932Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T08:39:27.396 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:26 smithi131 bash[71048]: 82e0910de1da02fb6403d51d8142224b44f4329c3f18ceb62edba2e63aa546cf 2024-09-11T08:39:27.396 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:39:27 smithi131 systemd[1]: Started Ceph node-exporter.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:39:27.464 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:39:27.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:27 smithi189 ceph-mon[32675]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:27.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:27 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]': finished 2024-09-11T08:39:27.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:27 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-11T08:39:27.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:27 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-11T08:39:27.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:27 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-09-11T08:39:27.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:27 smithi189 ceph-mon[32675]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:39:27.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:27 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:27.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:27 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:27.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:27 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:39:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:39:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:39:27.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[37208]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:27.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]': finished 2024-09-11T08:39:27.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-11T08:39:27.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-11T08:39:27.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-09-11T08:39:27.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[37208]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:39:27.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:27.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:27.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:27.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[29446]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:27.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]': finished 2024-09-11T08:39:27.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-11T08:39:27.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-11T08:39:27.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-09-11T08:39:27.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[29446]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:39:27.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:27.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:27.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:27 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:28.240 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:27 smithi189 bash[63459]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-11T08:39:28.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:28 smithi189 ceph-mon[32675]: Deploying daemon node-exporter.b on smithi189 2024-09-11T08:39:28.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:28 smithi189 ceph-mon[32675]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:39:28.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:28 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3808728457' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:28.740 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:28 smithi189 bash[63459]: Getting image source signatures 2024-09-11T08:39:28.741 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:28 smithi189 bash[63459]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T08:39:28.741 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:28 smithi189 bash[63459]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-11T08:39:28.741 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:28 smithi189 bash[63459]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-11T08:39:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:28 smithi131 ceph-mon[29446]: Deploying daemon node-exporter.b on smithi189 2024-09-11T08:39:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:28 smithi131 ceph-mon[29446]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:39:28.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:28 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3808728457' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:28.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:28 smithi131 ceph-mon[37208]: Deploying daemon node-exporter.b on smithi189 2024-09-11T08:39:28.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:28 smithi131 ceph-mon[37208]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:39:28.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:28 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3808728457' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:29.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:29 smithi189 ceph-mon[32675]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:29.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:29 smithi189 ceph-mon[32675]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:39:29.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:39:29 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:39:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:39:29.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:29 smithi131 ceph-mon[29446]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:29.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:29 smithi131 ceph-mon[29446]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:39:29.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:29 smithi131 ceph-mon[37208]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:29.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:29 smithi131 ceph-mon[37208]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:39:30.240 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:29 smithi189 bash[63459]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T08:39:30.240 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 bash[63459]: Writing manifest to image destination 2024-09-11T08:39:30.706 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 podman[63459]: 2024-09-11 08:39:30.379269599 +0000 UTC m=+2.520626872 container create 4df34c86104491bfcf8f08866d3dfeefa652889703edc6a35fd308a027ea697c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:39:30.707 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 podman[63459]: 2024-09-11 08:39:30.2890538 +0000 UTC m=+2.430411088 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T08:39:30.707 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 podman[63459]: 2024-09-11 08:39:30.606374922 +0000 UTC m=+2.747732196 container init 4df34c86104491bfcf8f08866d3dfeefa652889703edc6a35fd308a027ea697c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:39:30.707 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 podman[63459]: 2024-09-11 08:39:30.609694587 +0000 UTC m=+2.751051855 container start 4df34c86104491bfcf8f08866d3dfeefa652889703edc6a35fd308a027ea697c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:39:30.707 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.613Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T08:39:30.707 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.613Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T08:39:30.707 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.614Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T08:39:30.707 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.614Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T08:39:30.707 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.614Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T08:39:30.707 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.614Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T08:39:30.707 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T08:39:30.707 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T08:39:30.707 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T08:39:30.707 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T08:39:30.707 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T08:39:30.708 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T08:39:30.708 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T08:39:30.708 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T08:39:30.708 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T08:39:30.708 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T08:39:30.708 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T08:39:30.708 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T08:39:30.709 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T08:39:30.709 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T08:39:30.709 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T08:39:30.709 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T08:39:30.709 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T08:39:30.709 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T08:39:30.709 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T08:39:30.709 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T08:39:30.709 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T08:39:30.709 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T08:39:30.709 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T08:39:30.709 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T08:39:30.709 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T08:39:30.709 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T08:39:30.710 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T08:39:30.710 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T08:39:30.710 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T08:39:30.710 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T08:39:30.710 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T08:39:30.710 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T08:39:30.710 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T08:39:30.710 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T08:39:30.710 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T08:39:30.710 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T08:39:30.710 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T08:39:30.710 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T08:39:30.710 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T08:39:30.710 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T08:39:30.711 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T08:39:30.711 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T08:39:30.711 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T08:39:30.711 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T08:39:30.711 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T08:39:30.711 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T08:39:30.711 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b[63536]: ts=2024-09-11T08:39:30.615Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T08:39:30.711 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 bash[63459]: 4df34c86104491bfcf8f08866d3dfeefa652889703edc6a35fd308a027ea697c 2024-09-11T08:39:30.990 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:39:30 smithi189 systemd[1]: Started Ceph node-exporter.b for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:39:31.444 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:39:31 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:39:31.726 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:31 smithi131 ceph-mon[37208]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:31.727 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:31 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.727 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:31 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.727 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:31 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.727 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:31 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.727 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:31 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.727 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:31 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:39:31.727 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:31 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:31 smithi131 ceph-mon[29446]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:31.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:31 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:31 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:31 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:31 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:31 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:31 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:39:31.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:31 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:31 smithi189 ceph-mon[32675]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:39:31.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:31 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:31 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:31 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:31 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:31 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:31.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:31 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:39:31.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:31 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:32.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:32 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:32.772 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:39:32.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:32 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:32.773 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:32 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:33.103 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:39:33.104 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:39:33.104 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:39:33.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:33 smithi189 ceph-mon[32675]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85 B/s, 2 objects/s recovering 2024-09-11T08:39:33.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:33 smithi189 ceph-mon[32675]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:39:33.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:33 smithi189 ceph-mon[32675]: Cluster is now healthy 2024-09-11T08:39:33.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:33 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3446834217' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:33.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:33 smithi131 ceph-mon[29446]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85 B/s, 2 objects/s recovering 2024-09-11T08:39:33.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:33 smithi131 ceph-mon[29446]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:39:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:33 smithi131 ceph-mon[29446]: Cluster is now healthy 2024-09-11T08:39:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:33 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3446834217' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:33.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:33 smithi131 ceph-mon[37208]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85 B/s, 2 objects/s recovering 2024-09-11T08:39:33.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:33 smithi131 ceph-mon[37208]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:39:33.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:33 smithi131 ceph-mon[37208]: Cluster is now healthy 2024-09-11T08:39:33.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:33 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3446834217' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:34.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:34 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:34.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:34 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:35.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:34 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:35.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:34 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:34 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:34 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:35.971 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[29446]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85 B/s, 2 objects/s recovering 2024-09-11T08:39:35.971 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:35.971 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:35.971 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:39:35.971 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:39:35.972 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:35.972 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-11T08:39:35.972 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-11T08:39:35.972 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-11T08:39:35.972 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[37208]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85 B/s, 2 objects/s recovering 2024-09-11T08:39:35.972 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:35.972 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:35.972 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:39:35.972 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:39:35.972 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:35.972 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-11T08:39:35.973 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-11T08:39:35.973 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:35 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-11T08:39:35.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:35 smithi189 ceph-mon[32675]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85 B/s, 2 objects/s recovering 2024-09-11T08:39:35.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:35 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:35.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:35 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:35.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:35 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:39:35.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:35 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:39:35.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:35 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:35.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:35 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-11T08:39:35.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:35 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-11T08:39:35.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:35 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-11T08:39:36.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:36 smithi189 ceph-mon[32675]: Rotating authentication key for osd.0 2024-09-11T08:39:36.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:36 smithi189 ceph-mon[32675]: Reconfiguring daemon osd.0 on smithi131 2024-09-11T08:39:37.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:36 smithi131 ceph-mon[37208]: Rotating authentication key for osd.0 2024-09-11T08:39:37.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:36 smithi131 ceph-mon[37208]: Reconfiguring daemon osd.0 on smithi131 2024-09-11T08:39:37.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:36 smithi131 ceph-mon[29446]: Rotating authentication key for osd.0 2024-09-11T08:39:37.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:36 smithi131 ceph-mon[29446]: Reconfiguring daemon osd.0 on smithi131 2024-09-11T08:39:37.838 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:37 smithi131 ceph-mon[37208]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 70 B/s, 2 objects/s recovering 2024-09-11T08:39:37.838 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:37 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:37.838 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:37 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:37.838 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:37 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:37.838 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:37 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:37.838 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:37 smithi131 ceph-mon[29446]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 70 B/s, 2 objects/s recovering 2024-09-11T08:39:37.839 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:37 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:37.839 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:37 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:37.839 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:37 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:37.839 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:37 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:37.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:37 smithi189 ceph-mon[32675]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 70 B/s, 2 objects/s recovering 2024-09-11T08:39:37.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:37 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:37.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:37 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:37.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:37 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:37.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:37 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:38.106 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:39:38.413 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:39:38.413 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:39:38.413 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:39:38.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:38 smithi189 ceph-mon[32675]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:39:38.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:38 smithi189 ceph-mon[32675]: Reconfiguring daemon alertmanager.a on smithi131 2024-09-11T08:39:38.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:38 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1711347202' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:39.024 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:38 smithi131 ceph-mon[29446]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:39:39.024 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:38 smithi131 ceph-mon[29446]: Reconfiguring daemon alertmanager.a on smithi131 2024-09-11T08:39:39.024 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:38 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1711347202' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:39.024 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:38 smithi131 ceph-mon[37208]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:39:39.024 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:38 smithi131 ceph-mon[37208]: Reconfiguring daemon alertmanager.a on smithi131 2024-09-11T08:39:39.024 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:38 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1711347202' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:39.950 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:39 smithi131 ceph-mon[37208]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.6 KiB/s rd, 476 B/s wr, 11 op/s; 63 B/s, 2 objects/s recovering 2024-09-11T08:39:39.950 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:39 smithi131 ceph-mon[29446]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.6 KiB/s rd, 476 B/s wr, 11 op/s; 63 B/s, 2 objects/s recovering 2024-09-11T08:39:39.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:39 smithi189 ceph-mon[32675]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.6 KiB/s rd, 476 B/s wr, 11 op/s; 63 B/s, 2 objects/s recovering 2024-09-11T08:39:39.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:39:39 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:39:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:39:40.271 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:40 smithi131 systemd[1]: Stopping Ceph alertmanager.a for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:39:40.547 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:40 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[70160]: ts=2024-09-11T08:39:40.271Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:39:40.547 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:40 smithi131 podman[72177]: 2024-09-11 08:39:40.291946436 +0000 UTC m=+0.127752852 container died c239b7ed231e2cebff6db4aece7b20045a6b3e1ae0ba153d8631c5d58637e9f1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:40.890 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:40 smithi131 podman[72177]: 2024-09-11 08:39:40.738646962 +0000 UTC m=+0.574453420 container cleanup c239b7ed231e2cebff6db4aece7b20045a6b3e1ae0ba153d8631c5d58637e9f1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:40.890 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:40 smithi131 bash[72177]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T08:39:41.312 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 podman[72189]: 2024-09-11 08:39:41.006603824 +0000 UTC m=+0.710734824 container remove c239b7ed231e2cebff6db4aece7b20045a6b3e1ae0ba153d8631c5d58637e9f1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:41.313 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 podman[72189]: 2024-09-11 08:39:41.090705858 +0000 UTC m=+0.794836856 volume remove e36f8e69762ead5869f555fe0d7afbda835585b09f0964a9753046162c22f225 2024-09-11T08:39:41.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:39:41 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:39:41.635 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T08:39:41.635 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 systemd[1]: Stopped Ceph alertmanager.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:39:41.635 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 systemd[1]: Starting Ceph alertmanager.a for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:39:41.635 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 podman[72285]: 2024-09-11 08:39:41.443548056 +0000 UTC m=+0.015558915 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-11T08:39:41.635 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 podman[72285]: 2024-09-11 08:39:41.54131904 +0000 UTC m=+0.113329889 volume create da44ed3feab56fd10f01fd1d7fdab5a24367d0a1fde2655001c4033516cf25af 2024-09-11T08:39:41.635 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 podman[72285]: 2024-09-11 08:39:41.635667146 +0000 UTC m=+0.207677995 container create f1618f18eebadd5067d5707a97da15c8f66bf589c9f3bb76e33369ebf50ced19 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:41.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:41 smithi131 ceph-mon[37208]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s; 57 B/s, 1 objects/s recovering 2024-09-11T08:39:41.998 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:41 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:41.999 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:41 smithi131 ceph-mon[29446]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s; 57 B/s, 1 objects/s recovering 2024-09-11T08:39:41.999 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:41 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:42.008 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:41 smithi189 ceph-mon[32675]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s; 57 B/s, 1 objects/s recovering 2024-09-11T08:39:42.008 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:41 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:42.391 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 podman[72285]: 2024-09-11 08:39:41.921840087 +0000 UTC m=+0.493850936 container init f1618f18eebadd5067d5707a97da15c8f66bf589c9f3bb76e33369ebf50ced19 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:42.391 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 podman[72285]: 2024-09-11 08:39:41.924698961 +0000 UTC m=+0.496709813 container start f1618f18eebadd5067d5707a97da15c8f66bf589c9f3bb76e33369ebf50ced19 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:42.391 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:39:41.937Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-11T08:39:42.391 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:39:41.937Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-11T08:39:42.391 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:39:41.938Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.131 port=9094 2024-09-11T08:39:42.391 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:39:41.939Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-11T08:39:42.391 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:39:41.961Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:39:42.392 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:39:41.961Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:39:42.392 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:39:41.965Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-11T08:39:42.392 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:41 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:39:41.965Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-11T08:39:42.392 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:42 smithi131 bash[72285]: f1618f18eebadd5067d5707a97da15c8f66bf589c9f3bb76e33369ebf50ced19 2024-09-11T08:39:42.392 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:42 smithi131 systemd[1]: Started Ceph alertmanager.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:39:42.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:42 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:42.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:42 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:42.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:42 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:42.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:42 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:42.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:42 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:42.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:42 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:42.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:42 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:42.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:42 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:42.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:42 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:43.416 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:39:43.729 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:39:43.729 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:39:43.729 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:39:43.834 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:43 smithi189 ceph-mon[32675]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:39:43.834 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:43 smithi189 ceph-mon[32675]: Reconfiguring daemon prometheus.a on smithi189 2024-09-11T08:39:43.834 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:43 smithi189 ceph-mon[32675]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s; 57 B/s, 1 objects/s recovering 2024-09-11T08:39:44.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:43 smithi131 ceph-mon[29446]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:39:44.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:43 smithi131 ceph-mon[29446]: Reconfiguring daemon prometheus.a on smithi189 2024-09-11T08:39:44.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:43 smithi131 ceph-mon[29446]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s; 57 B/s, 1 objects/s recovering 2024-09-11T08:39:44.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:43 smithi131 ceph-mon[37208]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:39:44.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:43 smithi131 ceph-mon[37208]: Reconfiguring daemon prometheus.a on smithi189 2024-09-11T08:39:44.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:43 smithi131 ceph-mon[37208]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s; 57 B/s, 1 objects/s recovering 2024-09-11T08:39:44.141 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:43 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:39:43.940Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.001068169s 2024-09-11T08:39:44.932 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:44 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3326759855' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:45.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:44 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3326759855' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:45.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:44 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3326759855' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:45.240 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:45 smithi189 systemd[1]: Stopping Ceph prometheus.a for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:39:45.628 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:45 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:39:45.370Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:39:45.628 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:45 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:39:45.370Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T08:39:45.628 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:45 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:39:45.370Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T08:39:45.628 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:45 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:39:45.370Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T08:39:45.628 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:45 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:39:45.370Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T08:39:45.628 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:45 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:39:45.370Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T08:39:45.628 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:45 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:39:45.370Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T08:39:45.628 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:45 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:39:45.370Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T08:39:45.628 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:45 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:39:45.370Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T08:39:45.629 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:45 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:39:45.371Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T08:39:45.629 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:45 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:39:45.371Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T08:39:45.629 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:45 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[61334]: ts=2024-09-11T08:39:45.371Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T08:39:45.629 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:45 smithi189 podman[64436]: 2024-09-11 08:39:45.380515649 +0000 UTC m=+0.104914667 container died d75fb03a78480de74e2ccec511804f08fa233a2ed18d9e60e36180ec7d1a4812 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:45.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:45 smithi189 ceph-mon[32675]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:45.990 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:45 smithi189 podman[64436]: 2024-09-11 08:39:45.744882364 +0000 UTC m=+0.469281381 container cleanup d75fb03a78480de74e2ccec511804f08fa233a2ed18d9e60e36180ec7d1a4812 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:45.990 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:45 smithi189 bash[64436]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T08:39:46.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:45 smithi131 ceph-mon[37208]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:45 smithi131 ceph-mon[29446]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:46.438 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 podman[64448]: 2024-09-11 08:39:46.054341659 +0000 UTC m=+0.671638016 container remove d75fb03a78480de74e2ccec511804f08fa233a2ed18d9e60e36180ec7d1a4812 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:46.438 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-11T08:39:46.439 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 systemd[1]: Stopped Ceph prometheus.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:39:46.439 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.044s CPU time. 2024-09-11T08:39:46.439 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 systemd[1]: Starting Ceph prometheus.a for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:39:46.714 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 podman[64543]: 2024-09-11 08:39:46.342943588 +0000 UTC m=+0.017780917 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-11T08:39:46.715 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 podman[64543]: 2024-09-11 08:39:46.488522374 +0000 UTC m=+0.163359705 container create ba70272d94b088e09ec5a083b722f34eb6076db4a28662de528e2cd4e60ed9a3 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:46.991 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 podman[64543]: 2024-09-11 08:39:46.71571064 +0000 UTC m=+0.390547972 container init ba70272d94b088e09ec5a083b722f34eb6076db4a28662de528e2cd4e60ed9a3 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:46.991 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 podman[64543]: 2024-09-11 08:39:46.718812811 +0000 UTC m=+0.393650148 container start ba70272d94b088e09ec5a083b722f34eb6076db4a28662de528e2cd4e60ed9a3 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:39:46.991 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.743Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T08:39:46.991 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.743Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T08:39:46.991 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.743Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi189 (none))" 2024-09-11T08:39:46.991 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.743Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T08:39:46.991 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.743Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T08:39:46.991 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.745Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T08:39:46.991 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.745Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T08:39:46.991 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.746Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T08:39:46.992 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.746Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T08:39:46.992 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.749Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T08:39:46.992 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.749Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.849µs 2024-09-11T08:39:46.992 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.749Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T08:39:46.992 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.750Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-11T08:39:46.992 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.750Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-11T08:39:46.992 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.750Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=19.469µs wal_replay_duration=676.302µs wbl_replay_duration=103ns total_replay_duration=713.593µs 2024-09-11T08:39:46.992 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.751Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T08:39:46.992 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.751Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T08:39:46.992 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.751Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T08:39:46.992 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.770Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=19.499003ms db_storage=880ns remote_storage=1.158µs web_handler=418ns query_engine=677ns scrape=644.077µs scrape_sd=43.81µs notify=34.401µs notify_sd=13.071µs rules=18.377833ms tracing=8.442µs 2024-09-11T08:39:46.993 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.770Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T08:39:46.993 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:39:46.770Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T08:39:46.993 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 bash[64543]: ba70272d94b088e09ec5a083b722f34eb6076db4a28662de528e2cd4e60ed9a3 2024-09-11T08:39:46.993 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:39:46 smithi189 systemd[1]: Started Ceph prometheus.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:39:47.140 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:46 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:46] ENGINE Bus STOPPING 2024-09-11T08:39:47.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:47] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:39:47.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:47] ENGINE Bus STOPPED 2024-09-11T08:39:47.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:47] ENGINE Bus STARTING 2024-09-11T08:39:47.492 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:47] ENGINE Serving on http://:::9283 2024-09-11T08:39:47.492 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:47] ENGINE Bus STARTED 2024-09-11T08:39:47.492 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:47] ENGINE Bus STOPPING 2024-09-11T08:39:47.815 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[29446]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:47.815 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:47.815 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:47.815 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:39:47.815 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:39:47.815 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:47.815 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:39:47.815 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi189.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:39:47.815 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:47.815 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:39:47.815 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi189.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:39:47.816 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:47.816 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:39:47.816 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:47] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:39:47.816 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:47] ENGINE Bus STOPPED 2024-09-11T08:39:47.816 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:47] ENGINE Bus STARTING 2024-09-11T08:39:47.816 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:47] ENGINE Serving on http://:::9283 2024-09-11T08:39:47.816 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:47] ENGINE Bus STARTED 2024-09-11T08:39:47.816 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[37208]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:47.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:47.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:47.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:39:47.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:39:47.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:47.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:39:47.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi189.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:39:47.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:47.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:39:47.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi189.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:39:47.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:47.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:39:47.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:47 smithi189 ceph-mon[32675]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:47.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:47 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:47.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:47 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:47.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:47 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:39:47.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:47 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:39:47.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:47 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:47.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:47 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:39:47.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:47 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi189.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:39:47.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:47 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:47.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:47 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:39:47.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:47 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi189.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:39:47.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:47 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:47.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:47 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:39:48.140 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:47 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:47] ENGINE Bus STOPPING 2024-09-11T08:39:48.640 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:48] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:39:48.640 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:48] ENGINE Bus STOPPED 2024-09-11T08:39:48.641 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:48] ENGINE Bus STARTING 2024-09-11T08:39:48.641 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:48] ENGINE Serving on http://:::9283 2024-09-11T08:39:48.641 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:39:48] ENGINE Bus STARTED 2024-09-11T08:39:48.730 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:39:48.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:48 smithi189 ceph-mon[32675]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:39:48.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:48 smithi189 ceph-mon[32675]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:39:48.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:48 smithi189 ceph-mon[32675]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:39:48.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:48 smithi189 ceph-mon[32675]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi189.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:39:48.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:48 smithi189 ceph-mon[32675]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:39:48.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:48 smithi189 ceph-mon[32675]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi189.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:39:49.043 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:39:49.043 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:39:49.043 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:39:49.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-mon[37208]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:39:49.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-mon[37208]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:39:49.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-mon[37208]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:39:49.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-mon[37208]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi189.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:39:49.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-mon[37208]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:39:49.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-mon[37208]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi189.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:39:49.044 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-mon[29446]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:39:49.044 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-mon[29446]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:39:49.044 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-mon[29446]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:39:49.045 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-mon[29446]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi189.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:39:49.045 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-mon[29446]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:39:49.045 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:48 smithi131 ceph-mon[29446]: from='mon.? v1:172.21.15.189:0/1480703363' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi189.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:39:49.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:49 smithi189 ceph-mon[32675]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:49.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:49 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/664928090' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:50.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:49 smithi131 ceph-mon[37208]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:50.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:49 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/664928090' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:50.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:49 smithi131 ceph-mon[29446]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:50.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:49 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/664928090' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:51.112 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:50 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:51.112 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:50 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:51.113 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:50 smithi189 ceph-mon[32675]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:51.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:50 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:51.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:50 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:51.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:50 smithi131 ceph-mon[29446]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:51.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:50 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:51.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:50 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:51.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:50 smithi131 ceph-mon[37208]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:51.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:39:51 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:39:52.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:51 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:52.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:51 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:52.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:51 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:39:52.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:51 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:39:52.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:51 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:52.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:51 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:39:52.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:51 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:52.141 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:39:51 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:39:51.942Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003806561s 2024-09-11T08:39:52.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:51 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:52.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:51 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:52.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:51 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:39:52.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:51 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:39:52.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:51 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:52.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:51 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:39:52.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:51 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:52.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:51 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:52.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:51 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:52.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:51 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:39:52.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:51 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:39:52.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:51 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:52.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:51 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:39:52.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:51 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:39:53.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:52 smithi131 ceph-mon[29446]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:53.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:52 smithi131 ceph-mon[37208]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:53.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:52 smithi189 ceph-mon[32675]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:54.045 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:39:54.350 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:39:54.350 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:39:54.350 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:39:54.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:54 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:54.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:54 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:54.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:54 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1638062318' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:54.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:54 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:54.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:54 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:54.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:54 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1638062318' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:54.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:54 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:54.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:54 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:54.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:54 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1638062318' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:39:55.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:55 smithi131 ceph-mon[29446]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:55.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:55 smithi131 ceph-mon[37208]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:55.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:55 smithi189 ceph-mon[32675]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:56.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:56 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:56.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:56 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:56.931 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:56 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:39:57.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:57 smithi131 ceph-mon[37208]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:57.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:57 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:57.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:57 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:57.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:57 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:39:57.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:57 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:39:57.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:57 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:57 smithi131 ceph-mon[29446]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:57 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:57 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:57 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:39:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:57 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:39:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:57 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:57.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:57 smithi189 ceph-mon[32675]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:39:57.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:57 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:57.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:57 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:57.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:57 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:39:57.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:57 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:39:57.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:57 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:39:59.352 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:39:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:39:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:39:59.669 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:39:59.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:39:59 smithi131 ceph-mon[37208]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:59.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:39:59 smithi131 ceph-mon[29446]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:59.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:39:59 smithi189 ceph-mon[32675]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:39:59.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:39:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:39:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:00.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:40:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:40:00] "GET /metrics HTTP/1.1" 200 219804 "" "Prometheus/2.43.0" 2024-09-11T08:40:00.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:00 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/571972983' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:00.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:00 smithi131 ceph-mon[37208]: overall HEALTH_OK 2024-09-11T08:40:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:00 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/571972983' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:00 smithi131 ceph-mon[29446]: overall HEALTH_OK 2024-09-11T08:40:00.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:00 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/571972983' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:00.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:00 smithi189 ceph-mon[32675]: overall HEALTH_OK 2024-09-11T08:40:01.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:40:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:40:01.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:01 smithi131 ceph-mon[37208]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:01.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:01 smithi131 ceph-mon[29446]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:01.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:01 smithi189 ceph-mon[32675]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:02.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:02 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:02.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:02 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:02.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:02 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:03.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:03 smithi131 ceph-mon[37208]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:03.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:03 smithi131 ceph-mon[29446]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:03.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:03 smithi189 ceph-mon[32675]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:04.672 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:40:04.983 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:40:04.983 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:40:04.983 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:40:05.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:05 smithi131 ceph-mon[37208]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:05.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:05 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/4202326325' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:05.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:05 smithi131 ceph-mon[29446]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:05.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:05 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/4202326325' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:05.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:05 smithi189 ceph-mon[32675]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:05.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:05 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/4202326325' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:07.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:07 smithi131 ceph-mon[37208]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:07.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:07 smithi131 ceph-mon[29446]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:07.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:07 smithi189 ceph-mon[32675]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:09.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:09 smithi131 ceph-mon[37208]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:09 smithi131 ceph-mon[29446]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:09.985 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:40:09.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:09 smithi189 ceph-mon[32675]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:09.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:40:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:40:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:10.298 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:40:10.298 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:40:10.298 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:40:10.298 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:40:10 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:40:10] "GET /metrics HTTP/1.1" 200 219804 "" "Prometheus/2.43.0" 2024-09-11T08:40:10.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:10 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2882553146' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:10.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:10 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2882553146' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:10.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:10 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2882553146' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:11.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:40:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:40:11.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:11 smithi131 ceph-mon[29446]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:11 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:11.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:11 smithi131 ceph-mon[37208]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:11.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:11 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:11.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:11 smithi189 ceph-mon[32675]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:11.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:11 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:12.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:12 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:12.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:12 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:12.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:12 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:13.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:13 smithi189 ceph-mon[32675]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:14.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:13 smithi131 ceph-mon[37208]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:14.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:13 smithi131 ceph-mon[29446]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:15.299 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:40:15.611 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:40:15.612 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:40:15.612 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:40:15.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:15 smithi131 ceph-mon[37208]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:15.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:15 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/207085535' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:15 smithi131 ceph-mon[29446]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:15 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/207085535' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:15.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:15 smithi189 ceph-mon[32675]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:15.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:15 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/207085535' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:17.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:17 smithi189 ceph-mon[32675]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:18.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:17 smithi131 ceph-mon[37208]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:18.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:17 smithi131 ceph-mon[29446]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:19.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:19 smithi189 ceph-mon[32675]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:19.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:40:19 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:40:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:20.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:19 smithi131 ceph-mon[37208]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:20.035 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:19 smithi131 ceph-mon[29446]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:20.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:40:20 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:40:20] "GET /metrics HTTP/1.1" 200 219779 "" "Prometheus/2.43.0" 2024-09-11T08:40:20.613 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:40:20.924 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:40:20.925 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:40:20.925 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:40:21.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:40:21 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:40:21.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:21 smithi189 ceph-mon[32675]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T08:40:21.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:21 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1930331990' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:22.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:21 smithi131 ceph-mon[37208]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T08:40:22.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:21 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1930331990' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:22.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:21 smithi131 ceph-mon[29446]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T08:40:22.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:21 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1930331990' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:22.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:22 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:23.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:22 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:23.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:22 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:23.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:23 smithi189 ceph-mon[32675]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:24.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:23 smithi131 ceph-mon[37208]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:24.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:23 smithi131 ceph-mon[29446]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:25.927 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:25 smithi131 ceph-mon[29446]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T08:40:25.928 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:25 smithi131 ceph-mon[37208]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T08:40:25.928 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:40:25.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:25 smithi189 ceph-mon[32675]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T08:40:26.237 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:40:26.237 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:40:26.237 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:40:26.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:26 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/4121796636' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:26.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:26 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:27.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:26 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/4121796636' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:26 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:27.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:26 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/4121796636' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:27.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:26 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:27.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:27 smithi189 ceph-mon[32675]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T08:40:28.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:27 smithi131 ceph-mon[29446]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T08:40:28.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:27 smithi131 ceph-mon[37208]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T08:40:29.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:29 smithi189 ceph-mon[32675]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:29.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:40:29 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:40:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:30.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:29 smithi131 ceph-mon[37208]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:30.035 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:29 smithi131 ceph-mon[29446]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:30.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:40:30 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:40:30] "GET /metrics HTTP/1.1" 200 219776 "" "Prometheus/2.43.0" 2024-09-11T08:40:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:40:31.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:40:31 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:40:31.553 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:40:31.553 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:40:31.553 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:40:31.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:31 smithi131 ceph-mon[29446]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:31.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:31 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/573467343' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:31.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:31 smithi131 ceph-mon[37208]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:31.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:31 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/573467343' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:31.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:31 smithi189 ceph-mon[32675]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:31.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:31 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/573467343' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:32.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:32 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:33.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:32 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:33.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:32 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:33.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:33 smithi131 ceph-mon[37208]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:33.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:33 smithi131 ceph-mon[29446]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:33.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:33 smithi189 ceph-mon[32675]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:35.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:35 smithi189 ceph-mon[32675]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:36.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:35 smithi131 ceph-mon[37208]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:36.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:35 smithi131 ceph-mon[29446]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:36.555 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:40:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:40:36.866 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:40:36.866 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:40:37.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:37 smithi189 ceph-mon[32675]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:37.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:37 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3110756471' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:38.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:37 smithi131 ceph-mon[29446]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:37 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3110756471' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:38.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:37 smithi131 ceph-mon[37208]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:38.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:37 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3110756471' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:39.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:39 smithi189 ceph-mon[32675]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:39.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:40:39 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:40:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:40.034 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:39 smithi131 ceph-mon[29446]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:40.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:39 smithi131 ceph-mon[37208]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:40.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:40:40 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:40:40] "GET /metrics HTTP/1.1" 200 219776 "" "Prometheus/2.43.0" 2024-09-11T08:40:41.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:40:41 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:40:41.867 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:40:41.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:41 smithi189 ceph-mon[32675]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:41.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:41 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:42.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:41 smithi131 ceph-mon[37208]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:42.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:41 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:41 smithi131 ceph-mon[29446]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:41 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:42.180 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:40:42.180 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:40:42.181 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:40:42.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:42 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:42.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:42 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3969803050' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:43.021 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:42 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:43.021 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:42 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3969803050' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:43.022 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:42 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:43.022 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:42 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3969803050' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:43.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:43 smithi131 ceph-mon[37208]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:43 smithi131 ceph-mon[29446]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:43.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:43 smithi189 ceph-mon[32675]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:45.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:45 smithi189 ceph-mon[32675]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:46.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:45 smithi131 ceph-mon[37208]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:45 smithi131 ceph-mon[29446]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:47.182 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:40:47.496 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:40:47.496 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:40:47.496 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:40:47.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:47 smithi131 ceph-mon[37208]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:47.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:47 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2337762117' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:47.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:47 smithi131 ceph-mon[29446]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:47.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:47 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2337762117' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:47.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:47 smithi189 ceph-mon[32675]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:47.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:47 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2337762117' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:49.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:49 smithi189 ceph-mon[32675]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:49.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:40:49 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:40:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:40:50.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:49 smithi131 ceph-mon[37208]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:50.036 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:49 smithi131 ceph-mon[29446]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:50.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:40:50 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:40:50] "GET /metrics HTTP/1.1" 200 219775 "" "Prometheus/2.43.0" 2024-09-11T08:40:51.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:40:51 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:40:51.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:51 smithi189 ceph-mon[32675]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:52.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:51 smithi131 ceph-mon[29446]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:52.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:51 smithi131 ceph-mon[37208]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:52.498 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:40:52.810 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:40:52.810 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:40:52.810 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:40:52.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:52 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:52.811 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:52 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:52.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:52 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:40:53.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:53 smithi189 ceph-mon[32675]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:53.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:53 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/584793001' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:54.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:53 smithi131 ceph-mon[37208]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:54.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:53 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/584793001' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:54.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:53 smithi131 ceph-mon[29446]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:54.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:53 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/584793001' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:54.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:54 smithi189 ceph-mon[32675]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:55.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:54 smithi131 ceph-mon[37208]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:55.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:54 smithi131 ceph-mon[29446]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:56.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:56 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:56.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:56 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:56.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:56 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:40:57.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:57 smithi131 ceph-mon[29446]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:57.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:57 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:40:57.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:57 smithi131 ceph-mon[37208]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:57.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:57 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:40:57.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:57 smithi189 ceph-mon[32675]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:40:57.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:57 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:40:57.812 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:40:58.117 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:40:58.117 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:40:58.117 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:40:58.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:58 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:40:58.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:58 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:40:58.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:58 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:40:58.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:58 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/904725838' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:58.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:58 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:40:58.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:58 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:40:58.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:58 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:40:58.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:58 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/904725838' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:58.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:58 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:40:58.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:58 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:40:58.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:58 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:40:58.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:58 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/904725838' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:40:59.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:40:59 smithi189 ceph-mon[32675]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:40:59.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:40:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:40:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:00.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:40:59 smithi131 ceph-mon[37208]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:00.035 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:40:59 smithi131 ceph-mon[29446]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:00.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:41:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:41:00] "GET /metrics HTTP/1.1" 200 219780 "" "Prometheus/2.43.0" 2024-09-11T08:41:01.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:41:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:41:01.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:01 smithi189 ceph-mon[32675]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:02.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:01 smithi131 ceph-mon[37208]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:01 smithi131 ceph-mon[29446]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:02.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:02 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:03.119 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:41:03.119 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:02 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:03.120 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:02 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:03.447 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:41:03.447 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:41:03.447 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:41:03.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:03 smithi189 ceph-mon[32675]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:03.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:03 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3168992024' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:04.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:03 smithi131 ceph-mon[37208]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:04.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:03 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3168992024' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:04.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:03 smithi131 ceph-mon[29446]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:04.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:03 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3168992024' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:05.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:05 smithi189 ceph-mon[32675]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:06.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:05 smithi131 ceph-mon[37208]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:06.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:05 smithi131 ceph-mon[29446]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:07.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:07 smithi189 ceph-mon[32675]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:08.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:07 smithi131 ceph-mon[37208]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:08.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:07 smithi131 ceph-mon[29446]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:08.449 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:41:08.761 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:41:08.761 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:41:08.762 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:41:09.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:09 smithi189 ceph-mon[32675]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:09.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:09 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3785983934' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:09.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:41:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:41:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:10.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:09 smithi131 ceph-mon[37208]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:10.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:09 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3785983934' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:10.036 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:09 smithi131 ceph-mon[29446]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:10.036 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:09 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3785983934' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:10.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:41:10 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:41:10] "GET /metrics HTTP/1.1" 200 219780 "" "Prometheus/2.43.0" 2024-09-11T08:41:11.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:41:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:41:11.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:11 smithi189 ceph-mon[32675]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:11.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:11 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:12.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:11 smithi131 ceph-mon[29446]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:12.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:11 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:12.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:11 smithi131 ceph-mon[37208]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:12.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:11 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:12.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:12 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:13.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:12 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:13.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:12 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:13.763 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:41:13.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:13 smithi189 ceph-mon[32675]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:14.077 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:41:14.077 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:41:14.077 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:41:14.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:13 smithi131 ceph-mon[37208]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:13 smithi131 ceph-mon[29446]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:14.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:14 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/110100812' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:15.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:14 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/110100812' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:15.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:14 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/110100812' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:15.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:15 smithi189 ceph-mon[32675]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:16.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:15 smithi131 ceph-mon[37208]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:16.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:15 smithi131 ceph-mon[29446]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:17.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:17 smithi189 ceph-mon[32675]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:18.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:17 smithi131 ceph-mon[37208]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:18.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:17 smithi131 ceph-mon[29446]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:19.078 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:41:19.393 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:41:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:41:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:41:19.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:19 smithi189 ceph-mon[32675]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:19.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:19 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/918502584' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:19.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:41:19 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:41:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:20.034 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:19 smithi131 ceph-mon[29446]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:20.035 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:19 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/918502584' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:20.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:19 smithi131 ceph-mon[37208]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:20.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:19 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/918502584' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:20.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:41:20 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:41:20] "GET /metrics HTTP/1.1" 200 219775 "" "Prometheus/2.43.0" 2024-09-11T08:41:21.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:41:21 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:41:21.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:21 smithi189 ceph-mon[32675]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:22.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:21 smithi131 ceph-mon[29446]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:22.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:21 smithi131 ceph-mon[37208]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:22.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:22 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:23.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:22 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:23.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:22 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:23.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:23 smithi189 ceph-mon[32675]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:24.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:23 smithi131 ceph-mon[29446]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:24.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:23 smithi131 ceph-mon[37208]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:24.395 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:41:24.708 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:41:24.708 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:41:24.708 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:41:24.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:24 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2161134601' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:25.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:24 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2161134601' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:25.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:24 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2161134601' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:25.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:25 smithi189 ceph-mon[32675]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:26.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:25 smithi131 ceph-mon[37208]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:26.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:25 smithi131 ceph-mon[29446]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:27.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:26 smithi131 ceph-mon[37208]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:27.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:26 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:26 smithi131 ceph-mon[29446]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:26 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:27.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:26 smithi189 ceph-mon[32675]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:27.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:26 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:29.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:29 smithi131 ceph-mon[37208]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:29.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:29 smithi131 ceph-mon[29446]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:29.710 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:41:29.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:29 smithi189 ceph-mon[32675]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:29.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:41:29 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:41:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:30.016 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:41:30.016 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:41:30.017 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:41:30.307 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:41:30 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:41:30] "GET /metrics HTTP/1.1" 200 219773 "" "Prometheus/2.43.0" 2024-09-11T08:41:30.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:30 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1948939052' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:30.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:30 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1948939052' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:30.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:30 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1948939052' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:31.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:31 smithi189 ceph-mon[32675]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:31.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:41:31 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:41:31.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:31 smithi131 ceph-mon[29446]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:31.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:31 smithi131 ceph-mon[37208]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:32.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:32 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:32.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:32 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:32.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:32 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:33.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:33 smithi131 ceph-mon[29446]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:33.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:33 smithi131 ceph-mon[37208]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:33.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:33 smithi189 ceph-mon[32675]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:35.018 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:41:35.333 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:41:35.333 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:41:35.333 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:41:35.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:35 smithi131 ceph-mon[29446]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:35.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:35 smithi131 ceph-mon[37208]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:35.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:35 smithi189 ceph-mon[32675]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:36.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:36 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1730527817' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:36.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:36 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1730527817' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:36.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:36 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1730527817' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:37.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:37 smithi131 ceph-mon[29446]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:37.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:37 smithi131 ceph-mon[37208]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:37.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:37 smithi189 ceph-mon[32675]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:39.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:39 smithi131 ceph-mon[29446]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:39.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:39 smithi131 ceph-mon[37208]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:39.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:39 smithi189 ceph-mon[32675]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:39.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:41:39 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:41:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:41:40.335 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:41:40 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:41:40] "GET /metrics HTTP/1.1" 200 219773 "" "Prometheus/2.43.0" 2024-09-11T08:41:40.647 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:41:40.647 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:41:40.647 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:41:41.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:41 smithi189 ceph-mon[32675]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:41.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:41 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2228013421' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:41.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:41 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:41.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:41:41 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:41:41.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:41 smithi131 ceph-mon[37208]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:41.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:41 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2228013421' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:41.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:41 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:41.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:41 smithi131 ceph-mon[29446]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:41.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:41 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2228013421' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:41.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:41 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:42.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:42 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:42.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:42 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:42.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:42 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:43.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:43 smithi131 ceph-mon[37208]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:43.644 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:43 smithi131 ceph-mon[29446]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:43.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:43 smithi189 ceph-mon[32675]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:45.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:45 smithi131 ceph-mon[37208]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:45.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:45 smithi131 ceph-mon[29446]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:45.649 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:41:45.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:45 smithi189 ceph-mon[32675]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:45.958 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:41:45.958 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:41:45.958 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:41:46.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:46 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/933141323' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:46.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:46 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/933141323' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:46.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:46 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/933141323' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:47.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:47 smithi131 ceph-mon[29446]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:47.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:47 smithi131 ceph-mon[37208]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:47.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:47 smithi189 ceph-mon[32675]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:49.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:49 smithi131 ceph-mon[37208]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:49.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:49 smithi131 ceph-mon[29446]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:49.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:49 smithi189 ceph-mon[32675]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:49.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:41:49 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:41:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:41:50.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:41:50 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:41:50] "GET /metrics HTTP/1.1" 200 219726 "" "Prometheus/2.43.0" 2024-09-11T08:41:50.960 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:41:51.272 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:41:51.273 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:41:51.273 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:41:51.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:51 smithi131 ceph-mon[29446]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:51.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:51 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3082179564' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:51.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:51 smithi131 ceph-mon[37208]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:51.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:51 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3082179564' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:51.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:51 smithi189 ceph-mon[32675]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:51.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:51 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3082179564' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:51.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:41:51 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:41:52.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:52 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:52.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:52 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:52.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:52 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:41:53.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:53 smithi131 ceph-mon[37208]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:53.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:53 smithi131 ceph-mon[29446]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:53.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:53 smithi189 ceph-mon[32675]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:55.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:55 smithi131 ceph-mon[37208]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:55.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:55 smithi131 ceph-mon[29446]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:55.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:55 smithi189 ceph-mon[32675]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:56.274 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:41:56.584 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:41:56.585 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:41:56.585 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:41:56.585 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:56 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:56.585 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:56 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:56.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:56 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:41:57.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:57 smithi131 ceph-mon[29446]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:57.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:57 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/806285453' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:57.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:57 smithi131 ceph-mon[37208]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:57.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:57 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/806285453' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:57.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:57 smithi189 ceph-mon[32675]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:41:57.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:57 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/806285453' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:41:58.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:58 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:41:58.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:58 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:41:58.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:58 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:41:58.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:58 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:41:58.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:58 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:41:58.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:58 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:41:58.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:58 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:41:58.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:58 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:41:58.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:58 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:41:58.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:58 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:41:58.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:58 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:41:58.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:58 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:41:59.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:41:59 smithi131 ceph-mon[37208]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:59.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:41:59 smithi131 ceph-mon[29446]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:59.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:41:59 smithi189 ceph-mon[32675]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:41:59.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:41:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:41:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:00.391 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:42:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:42:00] "GET /metrics HTTP/1.1" 200 219720 "" "Prometheus/2.43.0" 2024-09-11T08:42:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:42:01.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:01 smithi131 ceph-mon[37208]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:01.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:01 smithi131 ceph-mon[29446]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:01.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:01 smithi189 ceph-mon[32675]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:01.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:42:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:42:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:42:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:42:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:42:02.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:02 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:02.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:02 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3799516500' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:02.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:02 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:02.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:02 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3799516500' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:02.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:02 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:02.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:02 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3799516500' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:03.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:03 smithi189 ceph-mon[32675]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:03.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:03 smithi131 ceph-mon[37208]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:03.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:03 smithi131 ceph-mon[29446]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:05.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:05 smithi189 ceph-mon[32675]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:05.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:05 smithi131 ceph-mon[37208]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:05.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:05 smithi131 ceph-mon[29446]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:06.900 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:42:07.215 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:42:07.215 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:42:07.215 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:42:07.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:07 smithi131 ceph-mon[37208]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:07.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:07 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1636374127' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:07.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:07 smithi131 ceph-mon[29446]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:07.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:07 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1636374127' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:07.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:07 smithi189 ceph-mon[32675]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:07.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:07 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1636374127' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:09.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:09 smithi189 ceph-mon[32675]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:09.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:42:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:42:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:09.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:09 smithi131 ceph-mon[37208]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:09.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:09 smithi131 ceph-mon[29446]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:10.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:42:10 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:42:10] "GET /metrics HTTP/1.1" 200 219720 "" "Prometheus/2.43.0" 2024-09-11T08:42:11.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:11 smithi189 ceph-mon[32675]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:11.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:11 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:11.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:42:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:42:11.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:11 smithi131 ceph-mon[37208]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:11.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:11 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:11 smithi131 ceph-mon[29446]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:11 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:12.216 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:42:12.535 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:42:12.535 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:42:12.536 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:42:12.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:12 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:12.536 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:12 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:12.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:12 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:13.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:13 smithi189 ceph-mon[32675]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:13.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:13 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3017638906' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:13.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:13 smithi131 ceph-mon[37208]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:13.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:13 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3017638906' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:13 smithi131 ceph-mon[29446]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:13 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3017638906' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:15.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:15 smithi189 ceph-mon[32675]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:15.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:15 smithi131 ceph-mon[37208]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:15 smithi131 ceph-mon[29446]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:17.537 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:42:17.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:17 smithi189 ceph-mon[32675]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:17.849 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:42:17.849 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:42:17.849 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:42:17.849 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:17 smithi131 ceph-mon[37208]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:17.850 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:17 smithi131 ceph-mon[29446]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:18.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:18 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/714627508' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:18.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:18 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/714627508' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:18.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:18 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/714627508' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:19.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:19 smithi189 ceph-mon[32675]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:19.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:42:19 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:42:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:19.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:19 smithi131 ceph-mon[37208]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:19.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:19 smithi131 ceph-mon[29446]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:20.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:42:20 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:42:20] "GET /metrics HTTP/1.1" 200 219730 "" "Prometheus/2.43.0" 2024-09-11T08:42:21.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:21 smithi189 ceph-mon[32675]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:21.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:42:21 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:42:21.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:21 smithi131 ceph-mon[29446]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:21.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:21 smithi131 ceph-mon[37208]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:22.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:22 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:22.850 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:42:22.851 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:22 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:22.851 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:22 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:23.166 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:42:23.166 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:42:23.166 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:42:23.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:23 smithi189 ceph-mon[32675]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:23.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:23 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2190409839' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:23.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:23 smithi131 ceph-mon[29446]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:23.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:23 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2190409839' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:23.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:23 smithi131 ceph-mon[37208]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:23.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:23 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2190409839' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:25.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:25 smithi189 ceph-mon[32675]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:25.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:25 smithi131 ceph-mon[29446]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:25.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:25 smithi131 ceph-mon[37208]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:26.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:26 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:26.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:26 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:26.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:26 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:27.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:27 smithi189 ceph-mon[32675]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:42:27.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:27 smithi131 ceph-mon[29446]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:42:27.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:27 smithi131 ceph-mon[37208]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:42:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:42:28.481 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:42:28.481 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:42:28.482 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:42:29.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:29 smithi189 ceph-mon[32675]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:29.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:29 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/887627122' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:29.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:42:29 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:42:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:29.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:29 smithi131 ceph-mon[29446]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:29.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:29 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/887627122' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:29.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:29 smithi131 ceph-mon[37208]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:29.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:29 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/887627122' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:30.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:42:30 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:42:30] "GET /metrics HTTP/1.1" 200 219734 "" "Prometheus/2.43.0" 2024-09-11T08:42:31.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:31 smithi189 ceph-mon[32675]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:42:31.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:42:31 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:42:31.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:31 smithi131 ceph-mon[29446]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:42:31.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:31 smithi131 ceph-mon[37208]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:42:32.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:32 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:32.792 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:32 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:32.792 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:32 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:42:33.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:33 smithi189 ceph-mon[32675]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:33.791 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:42:33.791 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:42:33.792 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:42:33.792 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:33 smithi131 ceph-mon[37208]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:33.792 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:33 smithi131 ceph-mon[29446]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:34.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:34 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/510370823' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:34.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:34 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/510370823' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:34.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:34 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/510370823' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:35.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:35 smithi189 ceph-mon[32675]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:42:35.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:35 smithi131 ceph-mon[29446]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:42:35.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:35 smithi131 ceph-mon[37208]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:42:37.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:37 smithi189 ceph-mon[32675]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:42:37.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:37 smithi131 ceph-mon[29446]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:42:37.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:37 smithi131 ceph-mon[37208]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:42:38.793 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:42:39.110 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:42:39.110 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:42:39.110 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:42:39.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:39 smithi189 ceph-mon[32675]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:39.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:39 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3062127501' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:39.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:42:39 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:42:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:39.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:39 smithi131 ceph-mon[29446]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:39.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:39 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3062127501' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:39.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:39 smithi131 ceph-mon[37208]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:39.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:39 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3062127501' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:40.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:42:40 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:42:40] "GET /metrics HTTP/1.1" 200 219734 "" "Prometheus/2.43.0" 2024-09-11T08:42:41.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:41 smithi189 ceph-mon[32675]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:41.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:41 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:41.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:42:41 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:42:41.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:41 smithi131 ceph-mon[37208]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:41.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:41 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:41.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:41 smithi131 ceph-mon[29446]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:41.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:41 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:42.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:42 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:42.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:42 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:42.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:42 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:43.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:43 smithi189 ceph-mon[32675]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:43.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:43 smithi131 ceph-mon[37208]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:43.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:43 smithi131 ceph-mon[29446]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:44.111 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:42:44.426 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:42:44.426 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:42:44.426 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:42:44.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:44 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3858527413' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:44.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:44 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3858527413' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:44.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:44 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3858527413' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:45.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:45 smithi131 ceph-mon[37208]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:45.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:45 smithi131 ceph-mon[29446]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:45.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:45 smithi189 ceph-mon[32675]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:47.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:47 smithi131 ceph-mon[37208]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:47.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:47 smithi131 ceph-mon[29446]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:47.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:47 smithi189 ceph-mon[32675]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:49.427 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:42:49.741 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:42:49.741 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:42:49.741 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:42:49.742 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:49 smithi131 ceph-mon[37208]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:49.742 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:49 smithi131 ceph-mon[29446]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:49.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:49 smithi189 ceph-mon[32675]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:49.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:42:49 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:42:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:42:50.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:42:50 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:42:50] "GET /metrics HTTP/1.1" 200 219743 "" "Prometheus/2.43.0" 2024-09-11T08:42:50.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:50 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/638370613' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:50.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:50 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/638370613' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:50.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:50 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/638370613' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:51.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:51 smithi189 ceph-mon[32675]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:51.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:42:51 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:42:51.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:51 smithi131 ceph-mon[29446]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:51.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:51 smithi131 ceph-mon[37208]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:52.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:52 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:52.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:52 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:52.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:52 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:42:53.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:53 smithi131 ceph-mon[29446]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:53.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:53 smithi131 ceph-mon[37208]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:53.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:53 smithi189 ceph-mon[32675]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:54.743 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:42:55.057 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:42:55.057 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:42:55.058 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:42:55.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:55 smithi131 ceph-mon[37208]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:55.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:55 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3424270469' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:55.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:55 smithi131 ceph-mon[29446]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:55.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:55 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3424270469' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:55.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:55 smithi189 ceph-mon[32675]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:55.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:55 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3424270469' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:42:56.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:56 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:56.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:56 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:56.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:56 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:42:57.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:57 smithi131 ceph-mon[37208]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:57 smithi131 ceph-mon[29446]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:57.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:57 smithi189 ceph-mon[32675]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:42:58.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:58 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:42:58.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:58 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:42:58.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:58 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:42:59.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:59 smithi131 ceph-mon[37208]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:59.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:59 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:42:59.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:59 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:42:59.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:42:59 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:42:59.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:59 smithi131 ceph-mon[29446]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:59.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:59 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:42:59.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:59 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:42:59.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:42:59 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:42:59.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:59 smithi189 ceph-mon[32675]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:42:59.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:59 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:42:59.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:59 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:42:59.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:42:59 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:42:59.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:42:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:42:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:00.059 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:43:00.371 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:43:00.371 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:43:00.371 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:43:00.372 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:43:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:43:00] "GET /metrics HTTP/1.1" 200 219745 "" "Prometheus/2.43.0" 2024-09-11T08:43:00.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:00 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/385404244' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:43:00.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:00 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/385404244' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:43:00.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:00 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/385404244' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:43:01.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:01 smithi189 ceph-mon[32675]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:01.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:43:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:43:01.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:01 smithi131 ceph-mon[29446]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:01.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:01 smithi131 ceph-mon[37208]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:02.792 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:02 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:02.792 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:02 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:02.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:02 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:03.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:03 smithi131 ceph-mon[37208]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:03.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:03 smithi131 ceph-mon[29446]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:03.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:03 smithi189 ceph-mon[32675]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:05.373 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:43:05.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:05 smithi131 ceph-mon[37208]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:05.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:05 smithi131 ceph-mon[29446]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:05.685 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:43:05.685 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:43:05.685 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:43:05.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:05 smithi189 ceph-mon[32675]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:06.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:06 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1107489831' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:43:06.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:06 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1107489831' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:43:06.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:06 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1107489831' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:43:07.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:07 smithi131 ceph-mon[37208]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:07.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:07 smithi131 ceph-mon[29446]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:07.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:07 smithi189 ceph-mon[32675]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:09.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:09 smithi131 ceph-mon[37208]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:09 smithi131 ceph-mon[29446]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:09.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:09 smithi189 ceph-mon[32675]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:09.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:43:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:43:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:10.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:43:10 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:43:10] "GET /metrics HTTP/1.1" 200 219745 "" "Prometheus/2.43.0" 2024-09-11T08:43:10.687 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:43:11.003 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== 2024-09-11T08:43:11.003 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== ']' 2024-09-11T08:43:11.003 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:43:11.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:43:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:43:11.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:11 smithi189 ceph-mon[32675]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:11.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:11 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/776929904' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:43:11.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:11 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:11 smithi131 ceph-mon[29446]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:11 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/776929904' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:43:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:11 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:11.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:11 smithi131 ceph-mon[37208]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:11.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:11 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/776929904' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:43:11.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:11 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:12.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:12 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:12.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:12 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:12.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:12 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:13.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:13 smithi131 ceph-mon[29446]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:13.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:13 smithi131 ceph-mon[37208]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:13.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:13 smithi189 ceph-mon[32675]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:15.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:15 smithi131 ceph-mon[29446]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:15.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:15 smithi131 ceph-mon[37208]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:15.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:15 smithi189 ceph-mon[32675]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:16.004 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.0 2024-09-11T08:43:16.316 INFO:teuthology.orchestra.run.smithi131.stdout:rotating key for osd.1 2024-09-11T08:43:16.317 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQBHV+FmrwXkFxAA+oBcPNexDUQpa7P7oq8QhA== 2024-09-11T08:43:16.317 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAdVuFmZjXfLRAAD6rJwFEokyD/CJsQ8eCQ1w== == AQBHV+FmrwXkFxAA+oBcPNexDUQpa7P7oq8QhA== ']' 2024-09-11T08:43:16.317 INFO:teuthology.orchestra.run.smithi131.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:43:16.317 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo 'rotating key for osd.1' 2024-09-11T08:43:16.317 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.1 2024-09-11T08:43:16.617 INFO:teuthology.orchestra.run.smithi131.stderr:+ K=AQA1VuFmullOKxAAynrpblnFjwAQvxlzjXojDA== 2024-09-11T08:43:16.618 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQA1VuFmullOKxAAynrpblnFjwAQvxlzjXojDA== 2024-09-11T08:43:16.618 INFO:teuthology.orchestra.run.smithi131.stderr:+ ceph orch daemon rotate-key osd.1 2024-09-11T08:43:16.874 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled to rotate-key osd.1 on host 'smithi131' 2024-09-11T08:43:16.874 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:16 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1575226078' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:43:16.874 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:16 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1575226078' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:43:16.885 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQA1VuFmullOKxAAynrpblnFjwAQvxlzjXojDA== == AQA1VuFmullOKxAAynrpblnFjwAQvxlzjXojDA== ']' 2024-09-11T08:43:16.885 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:43:16.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:16 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1575226078' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T08:43:17.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[29446]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3645310119' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:43:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:43:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:43:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:43:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-11T08:43:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-11T08:43:17.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-11T08:43:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[37208]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3645310119' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:43:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:43:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:43:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:43:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-11T08:43:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-11T08:43:17.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:17 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-11T08:43:17.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:17 smithi189 ceph-mon[32675]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:17.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:17 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3645310119' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:43:17.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:17 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:17.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:17 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:17.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:17 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:43:17.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:17 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:43:17.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:17 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:43:17.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:17 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:17.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:17 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-11T08:43:17.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:17 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-11T08:43:17.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:17 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-11T08:43:18.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:18 smithi131 ceph-mon[37208]: from='client.15036 v1:172.21.15.131:0/1311338994' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:43:18.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:18 smithi131 ceph-mon[37208]: Schedule rotate-key daemon osd.1 2024-09-11T08:43:18.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:18 smithi131 ceph-mon[37208]: Rotating authentication key for osd.1 2024-09-11T08:43:18.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:18 smithi131 ceph-mon[37208]: Reconfiguring daemon osd.1 on smithi131 2024-09-11T08:43:18.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:18 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:18.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:18 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:18.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:18 smithi131 ceph-mon[29446]: from='client.15036 v1:172.21.15.131:0/1311338994' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:43:18.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:18 smithi131 ceph-mon[29446]: Schedule rotate-key daemon osd.1 2024-09-11T08:43:18.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:18 smithi131 ceph-mon[29446]: Rotating authentication key for osd.1 2024-09-11T08:43:18.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:18 smithi131 ceph-mon[29446]: Reconfiguring daemon osd.1 on smithi131 2024-09-11T08:43:18.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:18 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:18.642 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:18 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:18.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:18 smithi189 ceph-mon[32675]: from='client.15036 v1:172.21.15.131:0/1311338994' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:43:18.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:18 smithi189 ceph-mon[32675]: Schedule rotate-key daemon osd.1 2024-09-11T08:43:18.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:18 smithi189 ceph-mon[32675]: Rotating authentication key for osd.1 2024-09-11T08:43:18.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:18 smithi189 ceph-mon[32675]: Reconfiguring daemon osd.1 on smithi131 2024-09-11T08:43:18.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:18 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:18.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:18 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:19.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:19 smithi189 ceph-mon[32675]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:19.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:19 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:19.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:19 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:19.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:43:19 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:43:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:20.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:19 smithi131 ceph-mon[37208]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:20.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:19 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:20.037 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:19 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:20.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:19 smithi131 ceph-mon[29446]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:20.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:19 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:20.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:19 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:20.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:43:20 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:43:20] "GET /metrics HTTP/1.1" 200 219735 "" "Prometheus/2.43.0" 2024-09-11T08:43:21.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:43:21 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:43:21.888 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.1 2024-09-11T08:43:21.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:21 smithi189 ceph-mon[32675]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:22.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:21 smithi131 ceph-mon[37208]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:22.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:21 smithi131 ceph-mon[29446]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:22.204 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQA1VuFmullOKxAAynrpblnFjwAQvxlzjXojDA== 2024-09-11T08:43:22.204 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQA1VuFmullOKxAAynrpblnFjwAQvxlzjXojDA== == AQA1VuFmullOKxAAynrpblnFjwAQvxlzjXojDA== ']' 2024-09-11T08:43:22.204 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:43:22.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:22 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:22.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:22 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/459456662' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:43:23.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:22 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:23.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:22 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/459456662' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:43:23.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:22 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:23.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:22 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/459456662' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:43:23.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:23 smithi189 ceph-mon[32675]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:24.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:23 smithi131 ceph-mon[37208]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:24.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:23 smithi131 ceph-mon[29446]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:25.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:25 smithi189 ceph-mon[32675]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:26.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:25 smithi131 ceph-mon[37208]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:26.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:25 smithi131 ceph-mon[29446]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:26.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:26 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:27.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:26 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:27.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:26 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:27.206 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.1 2024-09-11T08:43:27.519 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQA1VuFmullOKxAAynrpblnFjwAQvxlzjXojDA== 2024-09-11T08:43:27.519 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQA1VuFmullOKxAAynrpblnFjwAQvxlzjXojDA== == AQA1VuFmullOKxAAynrpblnFjwAQvxlzjXojDA== ']' 2024-09-11T08:43:27.520 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:43:27.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:27 smithi131 ceph-mon[29446]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:27.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:27 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1694881742' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:43:27.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:27 smithi131 ceph-mon[37208]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:27.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:27 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1694881742' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:43:27.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:27 smithi189 ceph-mon[32675]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:27.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:27 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1694881742' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:43:29.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:29 smithi189 ceph-mon[32675]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:29.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:43:29 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:43:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:30.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:29 smithi131 ceph-mon[37208]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:30.035 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:29 smithi131 ceph-mon[29446]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:30.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:43:30 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:43:30] "GET /metrics HTTP/1.1" 200 219741 "" "Prometheus/2.43.0" 2024-09-11T08:43:31.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:43:31 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:43:32.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:31 smithi131 ceph-mon[29446]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:32.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:31 smithi131 ceph-mon[37208]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:32.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:31 smithi189 ceph-mon[32675]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:32.522 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.1 2024-09-11T08:43:32.839 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQA1VuFmullOKxAAynrpblnFjwAQvxlzjXojDA== 2024-09-11T08:43:32.839 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQA1VuFmullOKxAAynrpblnFjwAQvxlzjXojDA== == AQA1VuFmullOKxAAynrpblnFjwAQvxlzjXojDA== ']' 2024-09-11T08:43:32.839 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:43:33.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:32 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:33.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:32 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:33.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:32 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:34.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:33 smithi131 ceph-mon[37208]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:34.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:33 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2265290062' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:43:34.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:33 smithi131 ceph-mon[29446]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:34.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:33 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2265290062' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:43:34.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:33 smithi189 ceph-mon[32675]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:34.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:33 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2265290062' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:43:36.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:35 smithi131 ceph-mon[37208]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:36.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:35 smithi131 ceph-mon[29446]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:36.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:35 smithi189 ceph-mon[32675]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:37.840 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.1 2024-09-11T08:43:38.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:37 smithi131 ceph-mon[29446]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:38.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:37 smithi131 ceph-mon[37208]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:38.149 INFO:teuthology.orchestra.run.smithi131.stdout:rotating key for osd.2 2024-09-11T08:43:38.149 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAkWOFmU8H4OBAAy5boI2QKj84jiS9Z23SAmA== 2024-09-11T08:43:38.149 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQA1VuFmullOKxAAynrpblnFjwAQvxlzjXojDA== == AQAkWOFmU8H4OBAAy5boI2QKj84jiS9Z23SAmA== ']' 2024-09-11T08:43:38.150 INFO:teuthology.orchestra.run.smithi131.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:43:38.150 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo 'rotating key for osd.2' 2024-09-11T08:43:38.150 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.2 2024-09-11T08:43:38.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:37 smithi189 ceph-mon[32675]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:38.452 INFO:teuthology.orchestra.run.smithi131.stderr:+ K=AQBNVuFmiHuNJhAAEiHbOwv6V0lO+GTZOfXdwA== 2024-09-11T08:43:38.452 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQBNVuFmiHuNJhAAEiHbOwv6V0lO+GTZOfXdwA== 2024-09-11T08:43:38.452 INFO:teuthology.orchestra.run.smithi131.stderr:+ ceph orch daemon rotate-key osd.2 2024-09-11T08:43:38.707 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled to rotate-key osd.2 on host 'smithi131' 2024-09-11T08:43:38.718 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQBNVuFmiHuNJhAAEiHbOwv6V0lO+GTZOfXdwA== == AQBNVuFmiHuNJhAAEiHbOwv6V0lO+GTZOfXdwA== ']' 2024-09-11T08:43:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:43:39.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:38 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1708106131' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:43:39.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:38 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/8047860' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:43:39.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:38 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:39.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:38 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:39.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:38 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:43:39.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:38 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1708106131' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:43:39.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:38 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/8047860' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:43:39.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:38 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:39.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:38 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:39.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:38 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:43:39.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:38 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1708106131' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T08:43:39.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:38 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/8047860' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:43:39.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:38 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:39.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:38 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:39.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:38 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:43:39.763 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:43:39 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:43:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:40.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[37208]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:40.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[37208]: from='client.15084 v1:172.21.15.131:0/1140529387' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:43:40.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[37208]: Schedule rotate-key daemon osd.2 2024-09-11T08:43:40.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:40.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:40.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:43:40.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:43:40.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:40.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[37208]: Rotating authentication key for osd.2 2024-09-11T08:43:40.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-11T08:43:40.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-11T08:43:40.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-11T08:43:40.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[37208]: Reconfiguring daemon osd.2 on smithi131 2024-09-11T08:43:40.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[29446]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:40.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[29446]: from='client.15084 v1:172.21.15.131:0/1140529387' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:43:40.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[29446]: Schedule rotate-key daemon osd.2 2024-09-11T08:43:40.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:40.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:40.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:43:40.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:43:40.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:40.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[29446]: Rotating authentication key for osd.2 2024-09-11T08:43:40.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-11T08:43:40.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-11T08:43:40.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-11T08:43:40.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:39 smithi131 ceph-mon[29446]: Reconfiguring daemon osd.2 on smithi131 2024-09-11T08:43:40.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:39 smithi189 ceph-mon[32675]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:40.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:39 smithi189 ceph-mon[32675]: from='client.15084 v1:172.21.15.131:0/1140529387' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:43:40.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:39 smithi189 ceph-mon[32675]: Schedule rotate-key daemon osd.2 2024-09-11T08:43:40.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:39 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:40.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:39 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:40.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:39 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:43:40.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:39 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:43:40.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:39 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:40.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:39 smithi189 ceph-mon[32675]: Rotating authentication key for osd.2 2024-09-11T08:43:40.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:39 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-11T08:43:40.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:39 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-11T08:43:40.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:39 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-11T08:43:40.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:39 smithi189 ceph-mon[32675]: Reconfiguring daemon osd.2 on smithi131 2024-09-11T08:43:40.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:43:40 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:43:40] "GET /metrics HTTP/1.1" 200 219741 "" "Prometheus/2.43.0" 2024-09-11T08:43:41.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:43:41 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:43:41.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:41 smithi189 ceph-mon[32675]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:41.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:41 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:41.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:41 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:41.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:41 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:41.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:41 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:41.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:41 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:41.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:41 smithi131 ceph-mon[29446]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:41.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:41 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:41.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:41 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:41.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:41 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:41.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:41 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:41.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:41 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:41.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:41 smithi131 ceph-mon[37208]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:41.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:41 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:41.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:41 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:41.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:41 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:41.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:41 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:43:41.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:41 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:42.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:42 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:42.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:42 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:42.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:42 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:43.721 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.2 2024-09-11T08:43:43.721 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:43 smithi131 ceph-mon[37208]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:43.721 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:43 smithi131 ceph-mon[29446]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:43.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:43 smithi189 ceph-mon[32675]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:44.031 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQBNVuFmiHuNJhAAEiHbOwv6V0lO+GTZOfXdwA== 2024-09-11T08:43:44.032 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQBNVuFmiHuNJhAAEiHbOwv6V0lO+GTZOfXdwA== == AQBNVuFmiHuNJhAAEiHbOwv6V0lO+GTZOfXdwA== ']' 2024-09-11T08:43:44.032 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:43:44.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:44 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2530692196' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:43:44.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:44 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2530692196' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:43:44.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:44 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2530692196' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:43:45.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:45 smithi131 ceph-mon[37208]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:45.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:45 smithi131 ceph-mon[29446]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:45.891 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:45 smithi189 ceph-mon[32675]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:47.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:47 smithi131 ceph-mon[37208]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:47.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:47 smithi131 ceph-mon[29446]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:47.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:47 smithi189 ceph-mon[32675]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:49.033 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.2 2024-09-11T08:43:49.346 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQBNVuFmiHuNJhAAEiHbOwv6V0lO+GTZOfXdwA== 2024-09-11T08:43:49.346 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQBNVuFmiHuNJhAAEiHbOwv6V0lO+GTZOfXdwA== == AQBNVuFmiHuNJhAAEiHbOwv6V0lO+GTZOfXdwA== ']' 2024-09-11T08:43:49.346 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:43:49.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:49 smithi131 ceph-mon[37208]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:49.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:49 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2295571620' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:43:49.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:49 smithi131 ceph-mon[29446]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:49.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:49 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2295571620' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:43:49.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:49 smithi189 ceph-mon[32675]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:49.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:49 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2295571620' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:43:49.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:43:49 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:43:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:43:50.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:43:50 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:43:50] "GET /metrics HTTP/1.1" 200 219747 "" "Prometheus/2.43.0" 2024-09-11T08:43:51.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:51 smithi189 ceph-mon[32675]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:51.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:43:51 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:43:51.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:51 smithi131 ceph-mon[37208]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:51.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:51 smithi131 ceph-mon[29446]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:52.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:52 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:52.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:52 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:52.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:52 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:43:53.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:53 smithi131 ceph-mon[37208]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:53.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:53 smithi131 ceph-mon[29446]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:53.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:53 smithi189 ceph-mon[32675]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:54.348 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.2 2024-09-11T08:43:54.663 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQBNVuFmiHuNJhAAEiHbOwv6V0lO+GTZOfXdwA== 2024-09-11T08:43:54.664 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQBNVuFmiHuNJhAAEiHbOwv6V0lO+GTZOfXdwA== == AQBNVuFmiHuNJhAAEiHbOwv6V0lO+GTZOfXdwA== ']' 2024-09-11T08:43:54.664 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:43:55.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:55 smithi131 ceph-mon[29446]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:55.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:55 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2486343845' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:43:55.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:55 smithi131 ceph-mon[37208]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:55.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:55 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2486343845' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:43:55.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:55 smithi189 ceph-mon[32675]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:55.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:55 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2486343845' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:43:56.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:56 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:56.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:56 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:56.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:56 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:43:57.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:57 smithi131 ceph-mon[29446]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:57.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:57 smithi131 ceph-mon[37208]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:57.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:57 smithi189 ceph-mon[32675]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:43:59.665 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.2 2024-09-11T08:43:59.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:43:59 smithi131 ceph-mon[37208]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:59.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:43:59 smithi131 ceph-mon[29446]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:59.977 INFO:teuthology.orchestra.run.smithi131.stdout:rotating key for osd.3 2024-09-11T08:43:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQA7WOFmeOt0BBAA/4gQUeC8Iey4G0QL1KSymQ== 2024-09-11T08:43:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQBNVuFmiHuNJhAAEiHbOwv6V0lO+GTZOfXdwA== == AQA7WOFmeOt0BBAA/4gQUeC8Iey4G0QL1KSymQ== ']' 2024-09-11T08:43:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:43:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo 'rotating key for osd.3' 2024-09-11T08:43:59.978 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.3 2024-09-11T08:43:59.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:43:59 smithi189 ceph-mon[32675]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:43:59.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:43:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:43:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:00.276 INFO:teuthology.orchestra.run.smithi131.stderr:+ K=AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== 2024-09-11T08:44:00.276 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== 2024-09-11T08:44:00.276 INFO:teuthology.orchestra.run.smithi131.stderr:+ ceph orch daemon rotate-key osd.3 2024-09-11T08:44:00.276 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:44:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:44:00] "GET /metrics HTTP/1.1" 200 219767 "" "Prometheus/2.43.0" 2024-09-11T08:44:00.531 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled to rotate-key osd.3 on host 'smithi131' 2024-09-11T08:44:00.545 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== == AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== ']' 2024-09-11T08:44:00.545 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:44:00.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:00 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1756241666' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:44:00.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:00 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/533899760' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:00.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:00 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:00.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:00 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:00.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:00 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:44:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:00 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1756241666' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:44:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:00 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/533899760' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:00 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:00 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:00 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:44:00.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:00 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1756241666' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T08:44:00.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:00 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/533899760' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:00.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:00 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:00.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:00 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:00.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:00 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:44:01.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:44:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:44:01.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:01 smithi189 ceph-mon[32675]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:01.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:01 smithi189 ceph-mon[32675]: from='client.25049 v1:172.21.15.131:0/326106208' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:44:01.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:01 smithi189 ceph-mon[32675]: Schedule rotate-key daemon osd.3 2024-09-11T08:44:01.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:01 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:01.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:01 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:01.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:01 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:44:01.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:01 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:44:01.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:01 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:01.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:01 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-11T08:44:01.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:01 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-11T08:44:01.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:01 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-11T08:44:01.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[29446]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:01.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[29446]: from='client.25049 v1:172.21.15.131:0/326106208' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:44:01.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[29446]: Schedule rotate-key daemon osd.3 2024-09-11T08:44:01.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:01.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:01.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:44:01.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:44:01.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:01.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-11T08:44:01.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-11T08:44:01.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-11T08:44:01.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[37208]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:01.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[37208]: from='client.25049 v1:172.21.15.131:0/326106208' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:44:01.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[37208]: Schedule rotate-key daemon osd.3 2024-09-11T08:44:01.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:01.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:01.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:44:01.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:44:01.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:01.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-11T08:44:01.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-11T08:44:01.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:01 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-11T08:44:02.660 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:02 smithi131 ceph-mon[37208]: Rotating authentication key for osd.3 2024-09-11T08:44:02.661 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:02 smithi131 ceph-mon[37208]: Reconfiguring daemon osd.3 on smithi131 2024-09-11T08:44:02.661 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:02 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:02.661 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:02 smithi131 ceph-mon[29446]: Rotating authentication key for osd.3 2024-09-11T08:44:02.661 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:02 smithi131 ceph-mon[29446]: Reconfiguring daemon osd.3 on smithi131 2024-09-11T08:44:02.661 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:02 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:02.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:02 smithi189 ceph-mon[32675]: Rotating authentication key for osd.3 2024-09-11T08:44:02.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:02 smithi189 ceph-mon[32675]: Reconfiguring daemon osd.3 on smithi131 2024-09-11T08:44:02.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:02 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:03.819 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:03 smithi131 ceph-mon[29446]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:03.819 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:03 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:03.819 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:03 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:03.819 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:03 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:03.819 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:03 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:03.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:03 smithi131 ceph-mon[37208]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:03.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:03 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:03.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:03 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:03.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:03 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:03.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:03 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:03.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:03 smithi189 ceph-mon[32675]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:03.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:03 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:03.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:03 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:03.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:03 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:03.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:03 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:05.548 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.3 2024-09-11T08:44:05.856 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:05 smithi131 ceph-mon[37208]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:05.857 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:05 smithi131 ceph-mon[29446]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:05.857 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== 2024-09-11T08:44:05.857 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== == AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== ']' 2024-09-11T08:44:05.857 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:44:05.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:05 smithi189 ceph-mon[32675]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:06.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:06 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2855729040' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:06.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:06 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2855729040' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:06.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:06 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2855729040' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:07.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:07 smithi131 ceph-mon[29446]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:07.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:07 smithi131 ceph-mon[37208]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:07.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:07 smithi189 ceph-mon[32675]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:09.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:09 smithi131 ceph-mon[29446]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:09.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:09 smithi131 ceph-mon[37208]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:09.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:09 smithi189 ceph-mon[32675]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:09.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:44:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:44:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:10.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:44:10 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:44:10] "GET /metrics HTTP/1.1" 200 219767 "" "Prometheus/2.43.0" 2024-09-11T08:44:10.858 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.3 2024-09-11T08:44:11.175 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== 2024-09-11T08:44:11.175 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== == AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== ']' 2024-09-11T08:44:11.176 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:44:11.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:44:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:44:11.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:11 smithi189 ceph-mon[32675]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:11.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:11 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/312303101' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:11.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:11 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:11.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:11 smithi131 ceph-mon[37208]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:11.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:11 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/312303101' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:11.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:11 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:11 smithi131 ceph-mon[29446]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:11 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/312303101' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:11.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:11 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:12.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:12 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:12 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:12.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:12 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:13.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:13 smithi131 ceph-mon[37208]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:13.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:13 smithi131 ceph-mon[29446]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:13.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:13 smithi189 ceph-mon[32675]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:15.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:15 smithi131 ceph-mon[37208]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:15.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:15 smithi131 ceph-mon[29446]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:15.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:15 smithi189 ceph-mon[32675]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:16.177 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.3 2024-09-11T08:44:16.488 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== 2024-09-11T08:44:16.489 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== == AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== ']' 2024-09-11T08:44:16.489 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:44:16.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:16 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2753579722' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:16.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:16 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2753579722' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:16.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:16 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2753579722' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:17.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:17 smithi131 ceph-mon[37208]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:17.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:17 smithi131 ceph-mon[29446]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:17.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:17 smithi189 ceph-mon[32675]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:19.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:19 smithi131 ceph-mon[37208]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:19.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:19 smithi131 ceph-mon[29446]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:19.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:19 smithi189 ceph-mon[32675]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:19.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:44:19 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:44:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:20.391 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:44:20 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:44:20] "GET /metrics HTTP/1.1" 200 219769 "" "Prometheus/2.43.0" 2024-09-11T08:44:21.491 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.3 2024-09-11T08:44:21.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:21 smithi189 ceph-mon[32675]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:21.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:44:21 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:44:21.804 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== 2024-09-11T08:44:21.804 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== == AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== ']' 2024-09-11T08:44:21.805 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:44:21.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:21 smithi131 ceph-mon[37208]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:21.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:21 smithi131 ceph-mon[29446]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:22.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:22 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:22.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:22 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/4108718795' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:22 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:22.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:22 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/4108718795' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:22.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:22 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:22.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:22 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/4108718795' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:23.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:23 smithi131 ceph-mon[37208]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:23.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:23 smithi131 ceph-mon[29446]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:23.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:23 smithi189 ceph-mon[32675]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:25.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:25 smithi131 ceph-mon[37208]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:25.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:25 smithi131 ceph-mon[29446]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:25.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:25 smithi189 ceph-mon[32675]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:26.806 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.3 2024-09-11T08:44:26.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:26 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:26.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:26 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:26.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:26 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:27.118 INFO:teuthology.orchestra.run.smithi131.stdout:rotating key for osd.4 2024-09-11T08:44:27.118 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQBRWOFmoqIzEBAAr5UkVmonC4PYZ6JCy4E3Gg== 2024-09-11T08:44:27.119 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQBmVuFm4oBgBRAAp5N1OoIQU7Po2vIn8aLWEQ== == AQBRWOFmoqIzEBAAr5UkVmonC4PYZ6JCy4E3Gg== ']' 2024-09-11T08:44:27.119 INFO:teuthology.orchestra.run.smithi131.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:44:27.119 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo 'rotating key for osd.4' 2024-09-11T08:44:27.119 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.4 2024-09-11T08:44:27.420 INFO:teuthology.orchestra.run.smithi131.stderr:+ K=AQB7VuFm0IRtKxAAqwm8WyZQsFQfyTtZooY7wQ== 2024-09-11T08:44:27.420 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQB7VuFm0IRtKxAAqwm8WyZQsFQfyTtZooY7wQ== 2024-09-11T08:44:27.420 INFO:teuthology.orchestra.run.smithi131.stderr:+ ceph orch daemon rotate-key osd.4 2024-09-11T08:44:27.676 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled to rotate-key osd.4 on host 'smithi189' 2024-09-11T08:44:27.677 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:27 smithi131 ceph-mon[37208]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:27.677 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:27 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/561608564' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:27.677 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:27 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2203895145' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:44:27.677 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:27 smithi131 ceph-mon[29446]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:27.677 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:27 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/561608564' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:27.677 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:27 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2203895145' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:44:27.688 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQB7VuFm0IRtKxAAqwm8WyZQsFQfyTtZooY7wQ== == AQB7VuFm0IRtKxAAqwm8WyZQsFQfyTtZooY7wQ== ']' 2024-09-11T08:44:27.688 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:44:27.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:27 smithi189 ceph-mon[32675]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:27.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:27 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/561608564' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T08:44:27.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:27 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2203895145' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:44:28.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:28 smithi189 ceph-mon[32675]: from='client.25103 v1:172.21.15.131:0/1021604612' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:44:28.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:28 smithi189 ceph-mon[32675]: Schedule rotate-key daemon osd.4 2024-09-11T08:44:28.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:28 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:28.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:28 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:28.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:28 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:44:28.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:28 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:44:28.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:28 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:44:28.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:28 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:28.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:28 smithi189 ceph-mon[32675]: Rotating authentication key for osd.4 2024-09-11T08:44:28.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:28 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-11T08:44:28.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:28 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-11T08:44:28.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:28 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-11T08:44:29.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[29446]: from='client.25103 v1:172.21.15.131:0/1021604612' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:44:29.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[29446]: Schedule rotate-key daemon osd.4 2024-09-11T08:44:29.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:29.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:29.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:44:29.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:44:29.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:44:29.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:29.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[29446]: Rotating authentication key for osd.4 2024-09-11T08:44:29.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-11T08:44:29.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-11T08:44:29.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-11T08:44:29.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[37208]: from='client.25103 v1:172.21.15.131:0/1021604612' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:44:29.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[37208]: Schedule rotate-key daemon osd.4 2024-09-11T08:44:29.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:29.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:29.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:44:29.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:44:29.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:44:29.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:29.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[37208]: Rotating authentication key for osd.4 2024-09-11T08:44:29.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-11T08:44:29.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-11T08:44:29.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:28 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-11T08:44:29.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:44:29 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:44:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:29.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:29 smithi189 ceph-mon[32675]: Reconfiguring daemon osd.4 on smithi189 2024-09-11T08:44:29.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:29 smithi189 ceph-mon[32675]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:29.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:29 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:29.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:29 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:29.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:29 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:29.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:29 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:30.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:29 smithi131 ceph-mon[37208]: Reconfiguring daemon osd.4 on smithi189 2024-09-11T08:44:30.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:29 smithi131 ceph-mon[37208]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:30.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:29 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:30.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:29 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:30.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:29 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:30.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:29 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:30.036 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:29 smithi131 ceph-mon[29446]: Reconfiguring daemon osd.4 on smithi189 2024-09-11T08:44:30.036 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:29 smithi131 ceph-mon[29446]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:30.036 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:29 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:30.036 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:29 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:30.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:29 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:30.037 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:29 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:30.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:44:30 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:44:30] "GET /metrics HTTP/1.1" 200 219770 "" "Prometheus/2.43.0" 2024-09-11T08:44:31.672 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:44:31 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:44:31.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:31 smithi189 ceph-mon[32675]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:32.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:31 smithi131 ceph-mon[37208]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:32.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:31 smithi131 ceph-mon[29446]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:32.690 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.4 2024-09-11T08:44:32.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:32 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQB7VuFm0IRtKxAAqwm8WyZQsFQfyTtZooY7wQ== 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQB7VuFm0IRtKxAAqwm8WyZQsFQfyTtZooY7wQ== == AQB7VuFm0IRtKxAAqwm8WyZQsFQfyTtZooY7wQ== ']' 2024-09-11T08:44:32.999 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:44:33.000 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:32 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:33.000 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:32 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:33.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:33 smithi189 ceph-mon[32675]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:33.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:33 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1842212844' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:44:34.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:33 smithi131 ceph-mon[29446]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:34.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:33 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1842212844' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:44:34.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:33 smithi131 ceph-mon[37208]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:34.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:33 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1842212844' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:44:35.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:35 smithi189 ceph-mon[32675]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:36.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:35 smithi131 ceph-mon[37208]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:36.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:35 smithi131 ceph-mon[29446]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:37.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:37 smithi189 ceph-mon[32675]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.4 2024-09-11T08:44:38.002 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:37 smithi131 ceph-mon[37208]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:38.002 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:37 smithi131 ceph-mon[29446]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQB7VuFm0IRtKxAAqwm8WyZQsFQfyTtZooY7wQ== 2024-09-11T08:44:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQB7VuFm0IRtKxAAqwm8WyZQsFQfyTtZooY7wQ== == AQB7VuFm0IRtKxAAqwm8WyZQsFQfyTtZooY7wQ== ']' 2024-09-11T08:44:38.313 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:44:38.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:38 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/4224200609' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:44:39.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:38 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/4224200609' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:44:39.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:38 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/4224200609' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:44:39.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:39 smithi189 ceph-mon[32675]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:39.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:44:39 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:44:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:40.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:39 smithi131 ceph-mon[37208]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:40.035 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:39 smithi131 ceph-mon[29446]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:40.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:44:40 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:44:40] "GET /metrics HTTP/1.1" 200 219770 "" "Prometheus/2.43.0" 2024-09-11T08:44:41.697 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:44:41 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:44:41.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:41 smithi189 ceph-mon[32675]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:41.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:41 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:42.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:41 smithi131 ceph-mon[37208]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:42.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:41 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:41 smithi131 ceph-mon[29446]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:42.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:41 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:42.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:42 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:43.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:42 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:43.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:42 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:43.315 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.4 2024-09-11T08:44:43.629 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQB7VuFm0IRtKxAAqwm8WyZQsFQfyTtZooY7wQ== 2024-09-11T08:44:43.629 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQB7VuFm0IRtKxAAqwm8WyZQsFQfyTtZooY7wQ== == AQB7VuFm0IRtKxAAqwm8WyZQsFQfyTtZooY7wQ== ']' 2024-09-11T08:44:43.629 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:44:43.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:43 smithi131 ceph-mon[29446]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:43.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:43 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/45637011' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:44:43.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:43 smithi131 ceph-mon[37208]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:43.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:43 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/45637011' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:44:43.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:43 smithi189 ceph-mon[32675]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:43.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:43 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/45637011' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:44:45.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:45 smithi189 ceph-mon[32675]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:46.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:45 smithi131 ceph-mon[37208]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:46.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:45 smithi131 ceph-mon[29446]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:47.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:47 smithi189 ceph-mon[32675]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:48.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:47 smithi131 ceph-mon[37208]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:48.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:47 smithi131 ceph-mon[29446]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:48.631 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.4 2024-09-11T08:44:48.943 INFO:teuthology.orchestra.run.smithi131.stdout:rotating key for osd.5 2024-09-11T08:44:48.944 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQBrWOFma8eeLRAAHWk0n0bMJ54RwhtPbGrGhA== 2024-09-11T08:44:48.944 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQB7VuFm0IRtKxAAqwm8WyZQsFQfyTtZooY7wQ== == AQBrWOFma8eeLRAAHWk0n0bMJ54RwhtPbGrGhA== ']' 2024-09-11T08:44:48.944 INFO:teuthology.orchestra.run.smithi131.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:44:48.944 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo 'rotating key for osd.5' 2024-09-11T08:44:48.944 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.5 2024-09-11T08:44:49.244 INFO:teuthology.orchestra.run.smithi131.stderr:+ K=AQCSVuFmHUmVBRAATqo85PJ5xLSmbtLFLNwMIw== 2024-09-11T08:44:49.244 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQCSVuFmHUmVBRAATqo85PJ5xLSmbtLFLNwMIw== 2024-09-11T08:44:49.245 INFO:teuthology.orchestra.run.smithi131.stderr:+ ceph orch daemon rotate-key osd.5 2024-09-11T08:44:49.499 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled to rotate-key osd.5 on host 'smithi189' 2024-09-11T08:44:49.511 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQCSVuFmHUmVBRAATqo85PJ5xLSmbtLFLNwMIw== == AQCSVuFmHUmVBRAATqo85PJ5xLSmbtLFLNwMIw== ']' 2024-09-11T08:44:49.511 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:44:49.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[37208]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:49.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1095212251' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:44:49.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/514299899' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:44:49.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:49.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:49.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:44:49.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:44:49.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:44:49.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:49.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-11T08:44:49.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-11T08:44:49.892 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-11T08:44:49.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[29446]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:49.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1095212251' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:44:49.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/514299899' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:44:49.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:49.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:49.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:44:49.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:44:49.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:44:49.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:49.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-11T08:44:49.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-11T08:44:49.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:49 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-11T08:44:49.954 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:44:49 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:44:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:44:49.954 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:49 smithi189 ceph-mon[32675]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:49.954 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:49 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1095212251' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T08:44:49.954 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:49 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/514299899' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:44:49.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:49 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:49.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:49 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:49.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:49 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:44:49.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:49 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:44:49.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:49 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:44:49.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:49 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:49.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:49 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-11T08:44:49.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:49 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-11T08:44:49.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:49 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-11T08:44:50.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:44:50 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:44:50] "GET /metrics HTTP/1.1" 200 219776 "" "Prometheus/2.43.0" 2024-09-11T08:44:50.924 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:50 smithi189 ceph-mon[32675]: from='client.15234 v1:172.21.15.131:0/2430740922' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:44:50.925 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:50 smithi189 ceph-mon[32675]: Schedule rotate-key daemon osd.5 2024-09-11T08:44:50.925 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:50 smithi189 ceph-mon[32675]: Rotating authentication key for osd.5 2024-09-11T08:44:50.925 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:50 smithi189 ceph-mon[32675]: Reconfiguring daemon osd.5 on smithi189 2024-09-11T08:44:51.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:50 smithi131 ceph-mon[37208]: from='client.15234 v1:172.21.15.131:0/2430740922' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:44:51.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:50 smithi131 ceph-mon[37208]: Schedule rotate-key daemon osd.5 2024-09-11T08:44:51.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:50 smithi131 ceph-mon[37208]: Rotating authentication key for osd.5 2024-09-11T08:44:51.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:50 smithi131 ceph-mon[37208]: Reconfiguring daemon osd.5 on smithi189 2024-09-11T08:44:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:50 smithi131 ceph-mon[29446]: from='client.15234 v1:172.21.15.131:0/2430740922' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:44:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:50 smithi131 ceph-mon[29446]: Schedule rotate-key daemon osd.5 2024-09-11T08:44:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:50 smithi131 ceph-mon[29446]: Rotating authentication key for osd.5 2024-09-11T08:44:51.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:50 smithi131 ceph-mon[29446]: Reconfiguring daemon osd.5 on smithi189 2024-09-11T08:44:51.733 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:44:51 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:44:51.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:51 smithi189 ceph-mon[32675]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:51.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:51 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:51.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:51 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:51.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:51 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:51.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:51 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:52.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:51 smithi131 ceph-mon[37208]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:52.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:51 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:52.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:51 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:52.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:51 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:52.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:51 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:52.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:51 smithi131 ceph-mon[29446]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:52.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:51 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:52.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:51 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:52.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:51 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:52.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:51 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:44:52.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:52 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:53.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:52 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:53.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:52 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:44:54.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:53 smithi131 ceph-mon[37208]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:54.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:53 smithi131 ceph-mon[29446]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:54.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:53 smithi189 ceph-mon[32675]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.5 2024-09-11T08:44:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQCSVuFmHUmVBRAATqo85PJ5xLSmbtLFLNwMIw== 2024-09-11T08:44:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQCSVuFmHUmVBRAATqo85PJ5xLSmbtLFLNwMIw== == AQCSVuFmHUmVBRAATqo85PJ5xLSmbtLFLNwMIw== ']' 2024-09-11T08:44:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:44:56.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:55 smithi131 ceph-mon[29446]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:56.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:55 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/4164399421' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:44:56.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:55 smithi131 ceph-mon[37208]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:56.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:55 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/4164399421' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:44:56.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:55 smithi189 ceph-mon[32675]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:56.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:55 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/4164399421' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:44:56.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:56 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:57.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:56 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:57.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:56 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:44:58.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:57 smithi131 ceph-mon[37208]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:58.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:57 smithi131 ceph-mon[29446]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:58.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:57 smithi189 ceph-mon[32675]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:44:59.825 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.5 2024-09-11T08:44:59.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:44:59 smithi189 ceph-mon[32675]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:44:59.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:44:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:44:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:00.035 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:44:59 smithi131 ceph-mon[37208]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:00.035 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:44:59 smithi131 ceph-mon[29446]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:00.138 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQCSVuFmHUmVBRAATqo85PJ5xLSmbtLFLNwMIw== 2024-09-11T08:45:00.138 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQCSVuFmHUmVBRAATqo85PJ5xLSmbtLFLNwMIw== == AQCSVuFmHUmVBRAATqo85PJ5xLSmbtLFLNwMIw== ']' 2024-09-11T08:45:00.138 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:45:00.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:45:00] "GET /metrics HTTP/1.1" 200 219771 "" "Prometheus/2.43.0" 2024-09-11T08:45:01.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:00 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3176553511' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:45:01.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:00 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3176553511' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:45:01.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:00 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3176553511' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:45:01.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:45:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:45:02.042 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:01 smithi131 ceph-mon[29446]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:02.042 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:01 smithi131 ceph-mon[37208]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:02.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:01 smithi189 ceph-mon[32675]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:03.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:02 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:03.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:02 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:03.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:02 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:04.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:03 smithi131 ceph-mon[37208]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:04.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:03 smithi131 ceph-mon[29446]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:04.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:03 smithi189 ceph-mon[32675]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:05.139 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.5 2024-09-11T08:45:05.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:04 smithi131 ceph-mon[37208]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:05.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:04 smithi131 ceph-mon[29446]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:05.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:04 smithi189 ceph-mon[32675]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:05.451 INFO:teuthology.orchestra.run.smithi131.stdout:rotating key for osd.6 2024-09-11T08:45:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQCBWOFml4XJIhAAmUx53JBg/WAq1V+gzd0smw== 2024-09-11T08:45:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQCSVuFmHUmVBRAATqo85PJ5xLSmbtLFLNwMIw== == AQCBWOFml4XJIhAAmUx53JBg/WAq1V+gzd0smw== ']' 2024-09-11T08:45:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:45:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo 'rotating key for osd.6' 2024-09-11T08:45:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.6 2024-09-11T08:45:05.754 INFO:teuthology.orchestra.run.smithi131.stderr:+ K=AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== 2024-09-11T08:45:05.754 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== 2024-09-11T08:45:05.754 INFO:teuthology.orchestra.run.smithi131.stderr:+ ceph orch daemon rotate-key osd.6 2024-09-11T08:45:06.013 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled to rotate-key osd.6 on host 'smithi189' 2024-09-11T08:45:06.013 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:05 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/341695518' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:45:06.013 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:05 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1612946756' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:06.013 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:05 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/341695518' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:45:06.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:05 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1612946756' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:06.024 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== == AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== ']' 2024-09-11T08:45:06.024 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:45:06.239 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:05 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/341695518' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T08:45:06.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:05 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1612946756' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:07.143 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:07 smithi189 ceph-mon[32675]: from='client.25193 v1:172.21.15.131:0/3628053583' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:45:07.144 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:07 smithi189 ceph-mon[32675]: Schedule rotate-key daemon osd.6 2024-09-11T08:45:07.144 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:07 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:07.144 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:07 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:07.144 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:07 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:07.144 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:07 smithi189 ceph-mon[32675]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:07.144 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:07 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:07.144 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:07 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:45:07.144 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:07 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:07.144 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:07 smithi189 ceph-mon[32675]: Rotating authentication key for osd.6 2024-09-11T08:45:07.144 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:07 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-11T08:45:07.144 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:07 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-11T08:45:07.144 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:07 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-11T08:45:07.145 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:07 smithi189 ceph-mon[32675]: Reconfiguring daemon osd.6 on smithi189 2024-09-11T08:45:07.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[29446]: from='client.25193 v1:172.21.15.131:0/3628053583' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:45:07.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[29446]: Schedule rotate-key daemon osd.6 2024-09-11T08:45:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[29446]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:45:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[29446]: Rotating authentication key for osd.6 2024-09-11T08:45:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-11T08:45:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-11T08:45:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-11T08:45:07.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[29446]: Reconfiguring daemon osd.6 on smithi189 2024-09-11T08:45:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[37208]: from='client.25193 v1:172.21.15.131:0/3628053583' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:45:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[37208]: Schedule rotate-key daemon osd.6 2024-09-11T08:45:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[37208]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:45:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[37208]: Rotating authentication key for osd.6 2024-09-11T08:45:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-11T08:45:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-11T08:45:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-11T08:45:07.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:07 smithi131 ceph-mon[37208]: Reconfiguring daemon osd.6 on smithi189 2024-09-11T08:45:09.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:08 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:09.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:08 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:09.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:08 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:09.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:08 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:09.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:08 smithi189 ceph-mon[32675]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:09.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:08 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:09.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:08 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:09.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:08 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:09.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:08 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:09.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:08 smithi131 ceph-mon[37208]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:09.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:08 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:09.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:08 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:09.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:08 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:09.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:08 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:09.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:08 smithi131 ceph-mon[29446]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:09.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:45:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:10.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:10 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:45:10] "GET /metrics HTTP/1.1" 200 219771 "" "Prometheus/2.43.0" 2024-09-11T08:45:11.026 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.6 2024-09-11T08:45:11.341 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== 2024-09-11T08:45:11.341 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== == AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== ']' 2024-09-11T08:45:11.341 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:45:11.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:11 smithi131 ceph-mon[37208]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:11.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:11 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:11.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:11 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/686719952' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:11.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:11 smithi131 ceph-mon[29446]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:11.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:11 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:11.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:11 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/686719952' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:11.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:45:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:45:11.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:11 smithi189 ceph-mon[32675]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:11.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:11 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:11.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:11 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/686719952' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:12.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:12 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:12.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:12 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:12.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:12 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:13.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:13 smithi131 ceph-mon[37208]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:13.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:13 smithi131 ceph-mon[29446]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:13.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:13 smithi189 ceph-mon[32675]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:15.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:15 smithi131 ceph-mon[37208]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:15.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:15 smithi131 ceph-mon[29446]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:15.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:15 smithi189 ceph-mon[32675]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:16.343 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.6 2024-09-11T08:45:16.654 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== 2024-09-11T08:45:16.654 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== == AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== ']' 2024-09-11T08:45:16.654 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:45:17.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:17 smithi131 ceph-mon[37208]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:17.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:17 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2199664531' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:17.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:17 smithi131 ceph-mon[29446]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:17.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:17 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2199664531' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:17.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:17 smithi189 ceph-mon[32675]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:17.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:17 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2199664531' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:19.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:19 smithi131 ceph-mon[37208]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:19.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:19 smithi131 ceph-mon[29446]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:19.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:19 smithi189 ceph-mon[32675]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:19.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:19 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:45:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:20.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:20 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:45:20] "GET /metrics HTTP/1.1" 200 219773 "" "Prometheus/2.43.0" 2024-09-11T08:45:21.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:21 smithi131 ceph-mon[37208]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:21.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:21 smithi131 ceph-mon[29446]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:21.656 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.6 2024-09-11T08:45:21.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:21 smithi189 ceph-mon[32675]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:21.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:45:21 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:45:21.966 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== 2024-09-11T08:45:21.966 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== == AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== ']' 2024-09-11T08:45:21.966 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:45:22.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:22 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:22.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:22 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2873946288' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:22.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:22 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:22.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:22 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2873946288' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:22.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:22 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:22.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:22 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2873946288' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:23.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:23 smithi131 ceph-mon[37208]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:23.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:23 smithi131 ceph-mon[29446]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:23.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:23 smithi189 ceph-mon[32675]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:25.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:25 smithi189 ceph-mon[32675]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:25.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:25 smithi131 ceph-mon[37208]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:25.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:25 smithi131 ceph-mon[29446]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:26.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:26 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:26.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:26 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:26.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:26 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:26.968 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.6 2024-09-11T08:45:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== 2024-09-11T08:45:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== == AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== ']' 2024-09-11T08:45:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:45:27.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:27 smithi131 ceph-mon[37208]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:27.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:27 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/731293290' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:27.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:27 smithi131 ceph-mon[29446]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:27.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:27 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/731293290' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:27.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:27 smithi189 ceph-mon[32675]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:27.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:27 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/731293290' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:29.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:29 smithi189 ceph-mon[32675]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:29.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:29 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:45:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:30.034 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:29 smithi131 ceph-mon[37208]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:30.035 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:29 smithi131 ceph-mon[29446]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:30.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:30 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:45:30] "GET /metrics HTTP/1.1" 200 219772 "" "Prometheus/2.43.0" 2024-09-11T08:45:31.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:31 smithi189 ceph-mon[32675]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:31.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:45:31 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:45:32.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:31 smithi131 ceph-mon[29446]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:32.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:31 smithi131 ceph-mon[37208]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:32.284 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.6 2024-09-11T08:45:32.594 INFO:teuthology.orchestra.run.smithi131.stdout:rotating key for osd.7 2024-09-11T08:45:32.595 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQCSWOFmhVmvGxAAy6Gq+Qb1X+8nRmG7PRUtLg== 2024-09-11T08:45:32.595 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQCpVuFmkjS9FhAAFu68Qh69tRz31KNr25rReQ== == AQCSWOFmhVmvGxAAy6Gq+Qb1X+8nRmG7PRUtLg== ']' 2024-09-11T08:45:32.595 INFO:teuthology.orchestra.run.smithi131.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:45:32.595 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo 'rotating key for osd.7' 2024-09-11T08:45:32.595 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.7 2024-09-11T08:45:32.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:32 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:32.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:32 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/176065513' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:32.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:32 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:32.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:32 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/176065513' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:32.899 INFO:teuthology.orchestra.run.smithi131.stderr:+ K=AQDAVuFm56TdDhAAEKVUB7SKJEWmw+oC4RjHFQ== 2024-09-11T08:45:32.899 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQDAVuFm56TdDhAAEKVUB7SKJEWmw+oC4RjHFQ== 2024-09-11T08:45:32.899 INFO:teuthology.orchestra.run.smithi131.stderr:+ ceph orch daemon rotate-key osd.7 2024-09-11T08:45:32.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:32 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:32.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:32 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/176065513' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T08:45:33.153 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled to rotate-key osd.7 on host 'smithi189' 2024-09-11T08:45:33.164 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQDAVuFm56TdDhAAEKVUB7SKJEWmw+oC4RjHFQ== == AQDAVuFm56TdDhAAEKVUB7SKJEWmw+oC4RjHFQ== ']' 2024-09-11T08:45:33.165 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:45:33.871 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:33 smithi189 ceph-mon[32675]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:33.871 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:33 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2061754472' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:45:33.871 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:33 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:33.871 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:33 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:33.871 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:33 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:33.871 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:33 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:33.871 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:33 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:45:33.871 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:33 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:33.872 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:33 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-11T08:45:33.872 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:33 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-11T08:45:33.872 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:33 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-11T08:45:34.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[37208]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:34.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2061754472' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:45:34.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:34.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:34.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:34.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:34.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:45:34.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:34.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-11T08:45:34.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-11T08:45:34.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-11T08:45:34.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[29446]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:34.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2061754472' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:45:34.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:34.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:34.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:34.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:34.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:45:34.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:34.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-11T08:45:34.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-11T08:45:34.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:33 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-11T08:45:34.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:34 smithi189 ceph-mon[32675]: from='client.25247 v1:172.21.15.131:0/1845795303' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:45:34.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:34 smithi189 ceph-mon[32675]: Schedule rotate-key daemon osd.7 2024-09-11T08:45:34.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:34 smithi189 ceph-mon[32675]: Rotating authentication key for osd.7 2024-09-11T08:45:34.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:34 smithi189 ceph-mon[32675]: Reconfiguring daemon osd.7 on smithi189 2024-09-11T08:45:35.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:34 smithi131 ceph-mon[37208]: from='client.25247 v1:172.21.15.131:0/1845795303' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:45:35.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:34 smithi131 ceph-mon[37208]: Schedule rotate-key daemon osd.7 2024-09-11T08:45:35.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:34 smithi131 ceph-mon[37208]: Rotating authentication key for osd.7 2024-09-11T08:45:35.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:34 smithi131 ceph-mon[37208]: Reconfiguring daemon osd.7 on smithi189 2024-09-11T08:45:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:34 smithi131 ceph-mon[29446]: from='client.25247 v1:172.21.15.131:0/1845795303' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:45:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:34 smithi131 ceph-mon[29446]: Schedule rotate-key daemon osd.7 2024-09-11T08:45:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:34 smithi131 ceph-mon[29446]: Rotating authentication key for osd.7 2024-09-11T08:45:35.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:34 smithi131 ceph-mon[29446]: Reconfiguring daemon osd.7 on smithi189 2024-09-11T08:45:36.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:35 smithi131 ceph-mon[37208]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:36.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:35 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:36.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:35 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:36.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:35 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:36.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:35 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:36.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:35 smithi131 ceph-mon[29446]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:36.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:35 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:36.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:35 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:36.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:35 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:36.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:35 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:36.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:35 smithi189 ceph-mon[32675]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:36.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:35 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:36.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:35 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:36.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:35 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:36.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:35 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:38.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:37 smithi131 ceph-mon[37208]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:38.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:37 smithi131 ceph-mon[29446]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:38.166 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.7 2024-09-11T08:45:38.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:37 smithi189 ceph-mon[32675]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:38.477 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQDAVuFm56TdDhAAEKVUB7SKJEWmw+oC4RjHFQ== 2024-09-11T08:45:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQDAVuFm56TdDhAAEKVUB7SKJEWmw+oC4RjHFQ== == AQDAVuFm56TdDhAAEKVUB7SKJEWmw+oC4RjHFQ== ']' 2024-09-11T08:45:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:45:39.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:38 smithi131 ceph-mon[29446]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:39.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:38 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1350058797' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:45:39.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:38 smithi131 ceph-mon[37208]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:39.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:38 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1350058797' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:45:39.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:38 smithi189 ceph-mon[32675]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:39.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:38 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1350058797' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:45:39.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:39 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:45:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:40.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:40 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:45:40] "GET /metrics HTTP/1.1" 200 219772 "" "Prometheus/2.43.0" 2024-09-11T08:45:41.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:41 smithi131 ceph-mon[37208]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:41.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:41 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:41.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:41 smithi131 ceph-mon[29446]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:41.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:41 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:41.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:41 smithi189 ceph-mon[32675]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:41.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:41 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:41.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:45:41 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:45:42.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:42 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:42.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:42 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:42.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:42 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:43.480 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.7 2024-09-11T08:45:43.640 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:43 smithi131 ceph-mon[37208]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:43.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:43 smithi131 ceph-mon[29446]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:43.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:43 smithi189 ceph-mon[32675]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:43.790 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQDAVuFm56TdDhAAEKVUB7SKJEWmw+oC4RjHFQ== 2024-09-11T08:45:43.791 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQDAVuFm56TdDhAAEKVUB7SKJEWmw+oC4RjHFQ== == AQDAVuFm56TdDhAAEKVUB7SKJEWmw+oC4RjHFQ== ']' 2024-09-11T08:45:43.791 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:45:44.641 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:44 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/2822529868' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:45:44.641 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:44 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/2822529868' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:45:44.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:44 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/2822529868' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:45:45.642 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:45 smithi189 ceph-mon[32675]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:45.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:45 smithi131 ceph-mon[37208]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:45.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:45 smithi131 ceph-mon[29446]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:47.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:47 smithi189 ceph-mon[32675]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:47.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:47 smithi131 ceph-mon[37208]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:47.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:47 smithi131 ceph-mon[29446]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:48.792 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.7 2024-09-11T08:45:49.103 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQDAVuFm56TdDhAAEKVUB7SKJEWmw+oC4RjHFQ== 2024-09-11T08:45:49.103 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQDAVuFm56TdDhAAEKVUB7SKJEWmw+oC4RjHFQ== == AQDAVuFm56TdDhAAEKVUB7SKJEWmw+oC4RjHFQ== ']' 2024-09-11T08:45:49.103 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:45:49.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:49 smithi189 ceph-mon[32675]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:49.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:49 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1925208834' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:45:49.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:49 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:45:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:45:49.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:49 smithi131 ceph-mon[29446]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:49.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:49 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1925208834' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:45:49.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:49 smithi131 ceph-mon[37208]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:49.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:49 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1925208834' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:45:50.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:50 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:45:50] "GET /metrics HTTP/1.1" 200 219778 "" "Prometheus/2.43.0" 2024-09-11T08:45:51.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:51 smithi189 ceph-mon[32675]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:51.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:45:51 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:45:51.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:51 smithi131 ceph-mon[37208]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:51.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:51 smithi131 ceph-mon[29446]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:52.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:52 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:52.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:52 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:52.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:52 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:45:53.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:53 smithi189 ceph-mon[32675]: pgmap v228: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:53.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:53 smithi131 ceph-mon[37208]: pgmap v228: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:53.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:53 smithi131 ceph-mon[29446]: pgmap v228: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:45:54.105 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key osd.7 2024-09-11T08:45:54.418 INFO:teuthology.orchestra.run.smithi131.stdout:rotating key for mgr.y 2024-09-11T08:45:54.419 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQCtWOFmL5NPDhAAT8U+JMURnwxF3xoYeZZioQ== 2024-09-11T08:45:54.419 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQDAVuFm56TdDhAAEKVUB7SKJEWmw+oC4RjHFQ== == AQCtWOFmL5NPDhAAT8U+JMURnwxF3xoYeZZioQ== ']' 2024-09-11T08:45:54.419 INFO:teuthology.orchestra.run.smithi131.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:45:54.419 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo 'rotating key for mgr.y' 2024-09-11T08:45:54.419 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key mgr.y 2024-09-11T08:45:54.722 INFO:teuthology.orchestra.run.smithi131.stderr:+ K=AQCwVeFmMBbvHBAAkzx1Q37DR6t8rbT5j8SkmA== 2024-09-11T08:45:54.722 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQCwVeFmMBbvHBAAkzx1Q37DR6t8rbT5j8SkmA== 2024-09-11T08:45:54.722 INFO:teuthology.orchestra.run.smithi131.stderr:+ ceph orch daemon rotate-key mgr.y 2024-09-11T08:45:54.722 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:54 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1460008199' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:45:54.723 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:54 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1460008199' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:45:54.981 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled to rotate-key mgr.y on host 'smithi131' 2024-09-11T08:45:54.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:54 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1460008199' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T08:45:54.993 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQCwVeFmMBbvHBAAkzx1Q37DR6t8rbT5j8SkmA== == AQCwVeFmMBbvHBAAkzx1Q37DR6t8rbT5j8SkmA== ']' 2024-09-11T08:45:54.993 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:45:55.728 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[37208]: pgmap v229: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:55.728 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/399560508' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T08:45:55.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:55.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:55.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:55.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:55.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:45:55.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:55.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:45:55.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:45:55.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T08:45:55.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:45:55.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:45:55.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:45:55.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:55.730 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[29446]: pgmap v229: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:55.730 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/399560508' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T08:45:55.730 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:55.730 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:55.730 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:55.730 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:55.730 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:45:55.730 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:55.730 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:45:55.730 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:45:55.730 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T08:45:55.730 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:45:55.731 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:45:55.731 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:45:55.731 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:55 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:55.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:55 smithi189 ceph-mon[32675]: pgmap v229: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:45:55.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:55 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/399560508' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T08:45:55.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:55 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:55.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:55 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:55.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:55 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:55.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:55 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:55.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:55 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:45:55.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:55 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:55.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:55 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:45:55.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:55 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:45:55.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:55 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T08:45:55.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:55 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:45:55.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:55 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:45:55.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:55 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:45:55.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:55 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:45:56.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-mon[37208]: from='client.25295 v1:172.21.15.131:0/3382396606' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:45:56.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-mon[37208]: Schedule rotate-key daemon mgr.y 2024-09-11T08:45:56.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-mon[37208]: Rotating authentication key for mgr.y 2024-09-11T08:45:56.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-mon[37208]: Reconfiguring daemon mgr.y on smithi131 2024-09-11T08:45:56.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:56.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:56.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:56.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-mon[29446]: from='client.25295 v1:172.21.15.131:0/3382396606' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:45:56.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-mon[29446]: Schedule rotate-key daemon mgr.y 2024-09-11T08:45:56.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-mon[29446]: Rotating authentication key for mgr.y 2024-09-11T08:45:56.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-mon[29446]: Reconfiguring daemon mgr.y on smithi131 2024-09-11T08:45:56.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:56.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:56.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:56.892 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.759+0000 7f6663a03640 -1 mgr.server reply reply (13) Permission denied access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication 2024-09-11T08:45:56.892 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.761+0000 7f6655967640 -1 log_channel(cephadm) log [ERR] : Non-zero return from ['ceph', '-k', '/var/lib/ceph/mgr/ceph-y/keyring', '-n', 'mgr.y', 'tell', 'mgr.y', 'rotate-key', '-i', '-']: 2024-09-11T08:45:56.755+0000 7fc229ffb640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 14128900569734243938 != 18374858748799134293 2024-09-11T08:45:56.893 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.756+0000 7fc2313b9640 1 -- 172.21.15.131:0/2427254699 >> v1:172.21.15.131:6790/0 conn(0x7fc22c10be00 legacy=0x7fc22c10e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:45:56.893 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.756+0000 7fc2313b9640 1 -- 172.21.15.131:0/2427254699 shutdown_connections 2024-09-11T08:45:56.893 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.756+0000 7fc2313b9640 1 -- 172.21.15.131:0/2427254699 >> 172.21.15.131:0/2427254699 conn(0x7fc22c06a6b0 msgr2=0x7fc22c06cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:45:56.895 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.756+0000 7fc2313b9640 1 -- 172.21.15.131:0/2427254699 shutdown_connections 2024-09-11T08:45:56.896 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.756+0000 7fc2313b9640 1 -- 172.21.15.131:0/2427254699 wait complete. 2024-09-11T08:45:56.896 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.756+0000 7fc2313b9640 1 Processor -- start 2024-09-11T08:45:56.896 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.756+0000 7fc2313b9640 1 -- start start 2024-09-11T08:45:56.896 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.756+0000 7fc2313b9640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22c11fff0 con 0x7fc22c10be00 2024-09-11T08:45:56.896 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.756+0000 7fc2313b9640 1 -- --> v1:172.21.15.189:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22c1211f0 con 0x7fc22c104710 2024-09-11T08:45:56.896 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.756+0000 7fc2313b9640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22c1223f0 con 0x7fc22c108210 2024-09-11T08:45:56.896 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.756+0000 7fc22a7fc640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7fc22c108210 0x7fc22c11ae90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:43968/0 (socket says 172.21.15.131:43968) 2024-09-11T08:45:56.896 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.756+0000 7fc22a7fc640 1 -- 172.21.15.131:0/3335522211 learned_addr learned my addr 172.21.15.131:0/3335522211 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-11T08:45:56.896 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.756+0000 7fc22b7fe640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc22c10be00 0x7fc22c11e6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56770/0 (socket says 172.21.15.131:56770) 2024-09-11T08:45:56.896 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2481701669 0 0) 0x7fc22c11fff0 con 0x7fc22c10be00 2024-09-11T08:45:56.896 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc214003120 con 0x7fc22c10be00 2024-09-11T08:45:56.896 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605932714 0 0) 0x7fc22c1223f0 con 0x7fc22c108210 2024-09-11T08:45:56.897 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc22c11fff0 con 0x7fc22c108210 2024-09-11T08:45:56.897 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 <== mon.1 v1:172.21.15.189:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326521722 0 0) 0x7fc22c1211f0 con 0x7fc22c104710 2024-09-11T08:45:56.897 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 --> v1:172.21.15.189:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc22c1223f0 con 0x7fc22c104710 2024-09-11T08:45:56.897 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 748+0+0 (unknown 3935940724 0 0) 0x7fc214003120 con 0x7fc22c10be00 2024-09-11T08:45:56.897 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc22c1211f0 con 0x7fc22c10be00 2024-09-11T08:45:56.897 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1834616200 0 0) 0x7fc224001670 con 0x7fc22c10be00 2024-09-11T08:45:56.897 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 748+0+0 (unknown 1355023089 0 0) 0x7fc22c11fff0 con 0x7fc22c108210 2024-09-11T08:45:56.897 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc214003120 con 0x7fc22c108210 2024-09-11T08:45:56.897 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1834616200 0 0) 0x7fc220001870 con 0x7fc22c108210 2024-09-11T08:45:56.897 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 <== mon.1 v1:172.21.15.189:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 748+0+0 (unknown 3881019650 0 0) 0x7fc22c1223f0 con 0x7fc22c104710 2024-09-11T08:45:56.897 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 --> v1:172.21.15.189:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc22c11fff0 con 0x7fc22c104710 2024-09-11T08:45:56.897 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 <== mon.1 v1:172.21.15.189:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1834616200 0 0) 0x7fc21c0019f0 con 0x7fc22c104710 2024-09-11T08:45:56.897 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 564+0+0 (unknown 1472153748 0 0) 0x7fc22c1211f0 con 0x7fc22c10be00 2024-09-11T08:45:56.898 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 >> v1:172.21.15.131:6790/0 conn(0x7fc22c108210 legacy=0x7fc22c11ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:45:56.898 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 >> v1:172.21.15.189:6789/0 conn(0x7fc22c104710 legacy=0x7fc22c117760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:45:56.898 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c1235f0 con 0x7fc22c10be00 2024-09-11T08:45:56.898 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 --> v1:172.21.15.131:6789/0 -- auth(proto 2 2 bytes epoch 0) v1 -- 0x7fc22c1223f0 con 0x7fc22c10be00 2024-09-11T08:45:56.898 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2313b9640 1 -- 172.21.15.131:0/3335522211 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22c1225e0 con 0x7fc22c10be00 2024-09-11T08:45:56.898 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2313b9640 1 -- 172.21.15.131:0/3335522211 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22c122ab0 con 0x7fc22c10be00 2024-09-11T08:45:56.898 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(37 keys) v1 ==== 1727+0+0 (unknown 3015570670 0 0) 0x7fc224001890 con 0x7fc22c10be00 2024-09-11T08:45:56.898 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1834616200 0 0) 0x7fc224006db0 con 0x7fc22c10be00 2024-09-11T08:45:56.898 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 <== mon.0 v1:172.21.15.131:6789/0 7 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 1162585381 0 0) 0x7fc22c122ab0 con 0x7fc22c10be00 2024-09-11T08:45:56.898 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.757+0000 7fc2137fe640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 14128900569734243938 != 18374858748799134293 2024-09-11T08:45:56.898 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.759+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 <== mon.0 v1:172.21.15.131:6789/0 8 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2156830833 0 0) 0x7fc224001d60 con 0x7fc22c10be00 2024-09-11T08:45:56.898 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.759+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 --> v1:172.21.15.131:6800/976762236 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc214073c10 con 0x7fc21406ff50 2024-09-11T08:45:56.898 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.759+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 <== mon.0 v1:172.21.15.131:6789/0 9 ==== osd_map(63..63 src has 1..63) v4 ==== 6284+0+0 (unknown 3207409118 0 0) 0x7fc22408a640 con 0x7fc22c10be00 2024-09-11T08:45:56.898 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.759+0000 7fc2137fe640 1 -- 172.21.15.131:0/3335522211 <== mgr.14583 v1:172.21.15.131:6800/976762236 1 ==== command_reply(tid 0: -13 access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication) v1 ==== 134+0+0 (unknown 3575743073 0 0) 0x7fc214073c10 con 0x7fc21406ff50 2024-09-11T08:45:56.899 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: Error EACCES: problem getting command descriptions from mgr.y 2024-09-11T08:45:56.899 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.899 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.773+0000 7f667a412640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-11T08:45:56.899 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ignoring --setuser ceph since I am not root 2024-09-11T08:45:56.899 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: ignoring --setgroup ceph since I am not root 2024-09-11T08:45:56.899 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-mgr[29677]: -- 172.21.15.131:0/1228401708 <== mon.0 v1:172.21.15.131:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 4238581156 0 0) 0x556063bceb60 con 0x556063bda800 2024-09-11T08:45:56.899 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-mgr[29677]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 14128900569734243938 != 18374858748799134293 2024-09-11T08:45:56.955 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:56 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: [11/Sep/2024:08:45:56] ENGINE Bus STOPPING 2024-09-11T08:45:56.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:56 smithi189 ceph-mon[32675]: from='client.25295 v1:172.21.15.131:0/3382396606' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:45:56.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:56 smithi189 ceph-mon[32675]: Schedule rotate-key daemon mgr.y 2024-09-11T08:45:56.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:56 smithi189 ceph-mon[32675]: Rotating authentication key for mgr.y 2024-09-11T08:45:56.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:56 smithi189 ceph-mon[32675]: Reconfiguring daemon mgr.y on smithi131 2024-09-11T08:45:56.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:56 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:56.955 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:56 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:56.956 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:56 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' 2024-09-11T08:45:57.218 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: [11/Sep/2024:08:45:57] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:45:57.219 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: [11/Sep/2024:08:45:57] ENGINE Bus STOPPED 2024-09-11T08:45:57.287 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.907+0000 7f7f32910ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:45:57.288 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:56 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:56.961+0000 7f7f32910ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:45:57.288 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:57.025+0000 7f7f32910ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:45:57.288 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:57.148+0000 7f7f32910ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:45:57.288 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:57.287+0000 7f7f32910ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:45:57.490 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: [11/Sep/2024:08:45:57] ENGINE Bus STARTING 2024-09-11T08:45:57.490 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: [11/Sep/2024:08:45:57] ENGINE Serving on http://:::9283 2024-09-11T08:45:57.490 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: [11/Sep/2024:08:45:57] ENGINE Bus STARTED 2024-09-11T08:45:57.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:57.548+0000 7f7f32910ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:45:57.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:57.611+0000 7f7f32910ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:45:57.891 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:57.790+0000 7f7f32910ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:45:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-11T08:45:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-11T08:45:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: Activating manager daemon x 2024-09-11T08:45:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:45:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-11T08:45:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: mgrmap e23: x(active, starting, since 0.00774246s) 2024-09-11T08:45:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:45:57.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:45:57.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:45:57.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:45:57.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:45:57.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:45:57.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:45:57.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:45:57.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:45:57.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:45:57.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:45:57.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:45:57.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:45:57.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:45:57.892 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:45:57.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: Manager daemon x is now available 2024-09-11T08:45:57.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:57.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:57.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:45:57.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:45:57.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T08:45:57.893 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T08:45:57.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-11T08:45:57.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-11T08:45:57.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: Activating manager daemon x 2024-09-11T08:45:57.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:45:57.893 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-11T08:45:57.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: mgrmap e23: x(active, starting, since 0.00774246s) 2024-09-11T08:45:57.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:45:57.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:45:57.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:45:57.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:45:57.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:45:57.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:45:57.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:45:57.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:45:57.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:45:57.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:45:57.894 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:45:57.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:45:57.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:45:57.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:45:57.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:45:57.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: Manager daemon x is now available 2024-09-11T08:45:57.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:57.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:57.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:45:57.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:45:57.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T08:45:57.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:57 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T08:45:57.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.14583 v1:172.21.15.131:0/2150016589' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-11T08:45:57.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-11T08:45:57.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: Activating manager daemon x 2024-09-11T08:45:57.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:45:57.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-11T08:45:57.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: mgrmap e23: x(active, starting, since 0.00774246s) 2024-09-11T08:45:57.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:45:57.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:45:57.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:45:57.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:45:57.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:45:57.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:45:57.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:45:57.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:45:57.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:45:57.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:45:57.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:45:57.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:45:57.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:45:57.991 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:45:57.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:45:57.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: Manager daemon x is now available 2024-09-11T08:45:57.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:45:57.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:45:57.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:45:57.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:45:57.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T08:45:57.992 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:57 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T08:45:58.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:45:58.429+0000 7f9faf229640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:45:58] ENGINE Error in HTTPServer.serve 2024-09-11T08:45:58.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: Traceback (most recent call last): 2024-09-11T08:45:58.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:45:58.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: self._connections.run(self.expiration_interval) 2024-09-11T08:45:58.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:45:58.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: self._run(expiration_interval) 2024-09-11T08:45:58.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:45:58.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:45:58.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:45:58.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:45:58.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:45:58.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: s = self.context.wrap_socket( 2024-09-11T08:45:58.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:45:58.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: return self.sslsocket_class._create( 2024-09-11T08:45:58.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:45:58.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: self.do_handshake() 2024-09-11T08:45:58.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:45:58.742 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: self._sslobj.do_handshake() 2024-09-11T08:45:58.742 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:45:58.742 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:45:59.139 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-mon[32675]: mgrmap e24: x(active, since 1.00893s) 2024-09-11T08:45:59.139 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-mon[32675]: [11/Sep/2024:08:45:58] ENGINE Bus STARTING 2024-09-11T08:45:59.139 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-mon[32675]: [11/Sep/2024:08:45:58] ENGINE Serving on https://172.21.15.189:7150 2024-09-11T08:45:59.139 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-mon[32675]: [11/Sep/2024:08:45:58] ENGINE Bus STARTED 2024-09-11T08:45:59.139 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-mon[32675]: [11/Sep/2024:08:45:58] ENGINE Error in HTTPServer.serve 2024-09-11T08:45:59.139 INFO:journalctl@ceph.mon.b.smithi189.stdout: Traceback (most recent call last): 2024-09-11T08:45:59.139 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:45:59.139 INFO:journalctl@ceph.mon.b.smithi189.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:45:59.139 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:45:59.140 INFO:journalctl@ceph.mon.b.smithi189.stdout: self._run(expiration_interval) 2024-09-11T08:45:59.140 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:45:59.140 INFO:journalctl@ceph.mon.b.smithi189.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:45:59.140 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:45:59.140 INFO:journalctl@ceph.mon.b.smithi189.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:45:59.140 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:45:59.140 INFO:journalctl@ceph.mon.b.smithi189.stdout: s = self.context.wrap_socket( 2024-09-11T08:45:59.140 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:45:59.140 INFO:journalctl@ceph.mon.b.smithi189.stdout: return self.sslsocket_class._create( 2024-09-11T08:45:59.140 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:45:59.140 INFO:journalctl@ceph.mon.b.smithi189.stdout: self.do_handshake() 2024-09-11T08:45:59.140 INFO:journalctl@ceph.mon.b.smithi189.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:45:59.140 INFO:journalctl@ceph.mon.b.smithi189.stdout: self._sslobj.do_handshake() 2024-09-11T08:45:59.140 INFO:journalctl@ceph.mon.b.smithi189.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:45:59.141 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:58 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:45:59.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:58 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:58.685+0000 7f7f32910ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:45:59.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:58 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:58.758+0000 7f7f32910ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:45:59.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:58 smithi131 ceph-mon[29446]: mgrmap e24: x(active, since 1.00893s) 2024-09-11T08:45:59.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:58 smithi131 ceph-mon[29446]: [11/Sep/2024:08:45:58] ENGINE Bus STARTING 2024-09-11T08:45:59.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:58 smithi131 ceph-mon[29446]: [11/Sep/2024:08:45:58] ENGINE Serving on https://172.21.15.189:7150 2024-09-11T08:45:59.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:58 smithi131 ceph-mon[29446]: [11/Sep/2024:08:45:58] ENGINE Bus STARTED 2024-09-11T08:45:59.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:58 smithi131 ceph-mon[29446]: [11/Sep/2024:08:45:58] ENGINE Error in HTTPServer.serve 2024-09-11T08:45:59.142 INFO:journalctl@ceph.mon.a.smithi131.stdout: Traceback (most recent call last): 2024-09-11T08:45:59.142 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:45:59.142 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:45:59.142 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:45:59.142 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._run(expiration_interval) 2024-09-11T08:45:59.142 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:45:59.142 INFO:journalctl@ceph.mon.a.smithi131.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:45:59.142 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:45:59.142 INFO:journalctl@ceph.mon.a.smithi131.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:45:59.142 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:45:59.142 INFO:journalctl@ceph.mon.a.smithi131.stdout: s = self.context.wrap_socket( 2024-09-11T08:45:59.142 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:45:59.143 INFO:journalctl@ceph.mon.a.smithi131.stdout: return self.sslsocket_class._create( 2024-09-11T08:45:59.143 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:45:59.143 INFO:journalctl@ceph.mon.a.smithi131.stdout: self.do_handshake() 2024-09-11T08:45:59.143 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:45:59.143 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._sslobj.do_handshake() 2024-09-11T08:45:59.143 INFO:journalctl@ceph.mon.a.smithi131.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:45:59.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:58 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:45:59.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:58 smithi131 ceph-mon[37208]: mgrmap e24: x(active, since 1.00893s) 2024-09-11T08:45:59.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:58 smithi131 ceph-mon[37208]: [11/Sep/2024:08:45:58] ENGINE Bus STARTING 2024-09-11T08:45:59.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:58 smithi131 ceph-mon[37208]: [11/Sep/2024:08:45:58] ENGINE Serving on https://172.21.15.189:7150 2024-09-11T08:45:59.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:58 smithi131 ceph-mon[37208]: [11/Sep/2024:08:45:58] ENGINE Bus STARTED 2024-09-11T08:45:59.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:58 smithi131 ceph-mon[37208]: [11/Sep/2024:08:45:58] ENGINE Error in HTTPServer.serve 2024-09-11T08:45:59.144 INFO:journalctl@ceph.mon.c.smithi131.stdout: Traceback (most recent call last): 2024-09-11T08:45:59.144 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:45:59.144 INFO:journalctl@ceph.mon.c.smithi131.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:45:59.144 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:45:59.144 INFO:journalctl@ceph.mon.c.smithi131.stdout: self._run(expiration_interval) 2024-09-11T08:45:59.144 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:45:59.144 INFO:journalctl@ceph.mon.c.smithi131.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:45:59.144 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:45:59.144 INFO:journalctl@ceph.mon.c.smithi131.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:45:59.144 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:45:59.144 INFO:journalctl@ceph.mon.c.smithi131.stdout: s = self.context.wrap_socket( 2024-09-11T08:45:59.144 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:45:59.144 INFO:journalctl@ceph.mon.c.smithi131.stdout: return self.sslsocket_class._create( 2024-09-11T08:45:59.144 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:45:59.144 INFO:journalctl@ceph.mon.c.smithi131.stdout: self.do_handshake() 2024-09-11T08:45:59.145 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:45:59.145 INFO:journalctl@ceph.mon.c.smithi131.stdout: self._sslobj.do_handshake() 2024-09-11T08:45:59.145 INFO:journalctl@ceph.mon.c.smithi131.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:45:59.145 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:58 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:45:59.784 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:59 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:59.513+0000 7f7f32910ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:45:59.784 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:59 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:59.630+0000 7f7f32910ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:45:59.784 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:59 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:59.755+0000 7f7f32910ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:45:59.784 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:45:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ::ffff:172.21.15.189 - - [11/Sep/2024:08:45:59] "GET /metrics HTTP/1.1" 200 34934 "" "Prometheus/2.43.0" 2024-09-11T08:45:59.994 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key mgr.y 2024-09-11T08:46:00.063 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:59 smithi131 ceph-mon[37208]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:46:00.063 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:45:59 smithi131 ceph-mon[37208]: mgrmap e25: x(active, since 2s) 2024-09-11T08:46:00.063 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:59 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:59.816+0000 7f7f32910ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:46:00.063 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:45:59 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:45:59.946+0000 7f7f32910ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:46:00.063 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:46:00.016+0000 7f7f32910ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:46:00.063 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:59 smithi131 ceph-mon[29446]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:46:00.063 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:45:59 smithi131 ceph-mon[29446]: mgrmap e25: x(active, since 2s) 2024-09-11T08:46:00.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:59 smithi189 ceph-mon[32675]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:46:00.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:45:59 smithi189 ceph-mon[32675]: mgrmap e25: x(active, since 2s) 2024-09-11T08:46:00.309 INFO:teuthology.orchestra.run.smithi131.stdout:rotating key for mgr.x 2024-09-11T08:46:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQDDWOFmsmXcAxAAhJ45NhI7Zet+sQNvOWpTog== 2024-09-11T08:46:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQCwVeFmMBbvHBAAkzx1Q37DR6t8rbT5j8SkmA== == AQDDWOFmsmXcAxAAhJ45NhI7Zet+sQNvOWpTog== ']' 2024-09-11T08:46:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T08:46:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo 'rotating key for mgr.x' 2024-09-11T08:46:00.310 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key mgr.x 2024-09-11T08:46:00.353 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:46:00.079+0000 7f7f32910ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:46:00.353 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:46:00.216+0000 7f7f32910ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:46:00.353 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:46:00.281+0000 7f7f32910ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:46:00.353 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:46:00.353+0000 7f7f32910ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:46:00.625 INFO:teuthology.orchestra.run.smithi131.stderr:+ K=AQAVVuFmu1GaLRAAQazMYhSUqEl3ss44SNbUYg== 2024-09-11T08:46:00.626 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAVVuFmu1GaLRAAQazMYhSUqEl3ss44SNbUYg== 2024-09-11T08:46:00.626 INFO:teuthology.orchestra.run.smithi131.stderr:+ ceph orch daemon rotate-key mgr.x 2024-09-11T08:46:00.626 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:46:00.440+0000 7f7f32910ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:46:00.626 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:46:00.501+0000 7f7f32910ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:46:00.626 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:00 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:46:00.562+0000 7f7f32910ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:46:00.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:00 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/837023400' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T08:46:00.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:00 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:00.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:00 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:00.891 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:00 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/3018315701' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:46:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:00 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/837023400' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T08:46:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:00 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:00 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:00.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:00 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/3018315701' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:46:00.899 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled to rotate-key mgr.x on host 'smithi189' 2024-09-11T08:46:00.911 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAVVuFmu1GaLRAAQazMYhSUqEl3ss44SNbUYg== == AQAVVuFmu1GaLRAAQazMYhSUqEl3ss44SNbUYg== ']' 2024-09-11T08:46:00.911 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:46:01.123 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:00 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/837023400' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T08:46:01.123 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:00 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:01.123 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:00 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:01.123 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:00 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/3018315701' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:46:01.640 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:46:01.337+0000 7f7f32910ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:46:01.641 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:46:01] ENGINE Bus STARTING 2024-09-11T08:46:01.641 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: CherryPy Checker: 2024-09-11T08:46:01.641 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: The Application mounted at '' has an empty config. 2024-09-11T08:46:01.641 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: 2024-09-11T08:46:01.641 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:46:01] ENGINE Serving on http://:::9283 2024-09-11T08:46:01.641 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[29673]: [11/Sep/2024:08:46:01] ENGINE Bus STARTED 2024-09-11T08:46:01.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:46:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:46:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: mgrmap e26: x(active, since 4s) 2024-09-11T08:46:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='client.25349 v1:172.21.15.131:0/162423336' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:46:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: Schedule rotate-key daemon mgr.x 2024-09-11T08:46:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: Adjusting osd_memory_target on smithi131 to 3248M 2024-09-11T08:46:02.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: Standby manager daemon y started 2024-09-11T08:46:02.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.? v1:172.21.15.131:0/3822079845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T08:46:02.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.? v1:172.21.15.131:0/3822079845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:46:02.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.? v1:172.21.15.131:0/3822079845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T08:46:02.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.? v1:172.21.15.131:0/3822079845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:46:02.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:02.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.143 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:46:02.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: mgrmap e26: x(active, since 4s) 2024-09-11T08:46:02.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='client.25349 v1:172.21.15.131:0/162423336' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:46:02.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: Schedule rotate-key daemon mgr.x 2024-09-11T08:46:02.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.143 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: Adjusting osd_memory_target on smithi131 to 3248M 2024-09-11T08:46:02.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: Standby manager daemon y started 2024-09-11T08:46:02.144 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.? v1:172.21.15.131:0/3822079845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T08:46:02.145 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.? v1:172.21.15.131:0/3822079845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:46:02.145 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.? v1:172.21.15.131:0/3822079845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T08:46:02.145 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.? v1:172.21.15.131:0/3822079845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:46:02.145 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:02.145 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.145 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:01 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:46:02.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: mgrmap e26: x(active, since 4s) 2024-09-11T08:46:02.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='client.25349 v1:172.21.15.131:0/162423336' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:46:02.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: Schedule rotate-key daemon mgr.x 2024-09-11T08:46:02.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:02.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: Adjusting osd_memory_target on smithi131 to 3248M 2024-09-11T08:46:02.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: Standby manager daemon y started 2024-09-11T08:46:02.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.? v1:172.21.15.131:0/3822079845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T08:46:02.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.? v1:172.21.15.131:0/3822079845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:46:02.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.? v1:172.21.15.131:0/3822079845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T08:46:02.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.? v1:172.21.15.131:0/3822079845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:46:02.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='client.14535 v1:172.21.15.189:0/2418365790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:46:02.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:02.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:01 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.450 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.450 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.450 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.451 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.451 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.451 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.451 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.451 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.451 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.451 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.451 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: Adjusting osd_memory_target on smithi189 to 3248M 2024-09-11T08:46:03.451 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.451 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:46:03.451 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:46:03.451 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-11T08:46:03.452 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-11T08:46:03.452 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: mgrmap e27: x(active, since 5s), standbys: y 2024-09-11T08:46:03.452 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:46:03.452 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:46:03.452 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:46:03.452 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:46:03.452 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:46:03.452 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.452 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.452 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.452 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.452 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.453 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:46:03.453 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:46:03.453 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T08:46:03.453 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:46:03.453 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:46:03.453 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:46:03.453 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:46:03.453 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.453 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.454 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.454 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.454 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.454 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.454 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.454 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.454 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.454 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.454 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: Adjusting osd_memory_target on smithi189 to 3248M 2024-09-11T08:46:03.454 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.454 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:46:03.454 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:46:03.454 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-11T08:46:03.454 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-11T08:46:03.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: mgrmap e27: x(active, since 5s), standbys: y 2024-09-11T08:46:03.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:46:03.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:46:03.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:46:03.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:46:03.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:46:03.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:46:03.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:46:03.455 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T08:46:03.456 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:46:03.456 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:46:03.456 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:46:03.456 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:03 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:46:03.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:46:03.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: Adjusting osd_memory_target on smithi189 to 3248M 2024-09-11T08:46:03.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:46:03.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:46:03.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-11T08:46:03.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-11T08:46:03.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: mgrmap e27: x(active, since 5s), standbys: y 2024-09-11T08:46:03.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:46:03.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:46:03.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:46:03.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:46:03.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:46:03.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:03.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:46:03.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T08:46:03.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T08:46:03.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:46:03.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:46:03.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:46:03.492 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:03 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:46:04.524 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:04 smithi131 ceph-mon[37208]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:46:04.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:04 smithi131 ceph-mon[37208]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:46:04.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:04 smithi131 ceph-mon[37208]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:46:04.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:04 smithi131 ceph-mon[37208]: Rotating authentication key for mgr.y 2024-09-11T08:46:04.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:04 smithi131 ceph-mon[37208]: Reconfiguring daemon mgr.y on smithi131 2024-09-11T08:46:04.525 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:04 smithi131 ceph-mon[29446]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:46:04.525 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:04 smithi131 ceph-mon[29446]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:46:04.525 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:04 smithi131 ceph-mon[29446]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:46:04.525 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:04 smithi131 ceph-mon[29446]: Rotating authentication key for mgr.y 2024-09-11T08:46:04.525 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:04 smithi131 ceph-mon[29446]: Reconfiguring daemon mgr.y on smithi131 2024-09-11T08:46:04.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:04 smithi189 ceph-mon[32675]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:46:04.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:04 smithi189 ceph-mon[32675]: Updating smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:46:04.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:04 smithi189 ceph-mon[32675]: Updating smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:46:04.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:04 smithi189 ceph-mon[32675]: Rotating authentication key for mgr.y 2024-09-11T08:46:04.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:04 smithi189 ceph-mon[32675]: Reconfiguring daemon mgr.y on smithi131 2024-09-11T08:46:05.465 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:05 smithi131 systemd[1]: Stopping Ceph mgr.y for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:46:05.821 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:05 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:05.821 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:05 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:05.822 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:05 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:05.822 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:05 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:05.822 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:05 smithi131 podman[79092]: 2024-09-11 08:46:05.466475614 +0000 UTC m=+0.110967234 container stop 353609dc3bc565043b3c1b060eb47c6f4881af5673056a4e0f5c5bf72adb7f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T08:46:05.822 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:05 smithi131 podman[79092]: 2024-09-11 08:46:05.492607334 +0000 UTC m=+0.137099000 container died 353609dc3bc565043b3c1b060eb47c6f4881af5673056a4e0f5c5bf72adb7f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T08:46:05.913 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key mgr.x 2024-09-11T08:46:05.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:05 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:05.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:05 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:06.122 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:05 smithi131 podman[79092]: 2024-09-11 08:46:05.930148269 +0000 UTC m=+0.574639889 container cleanup 353609dc3bc565043b3c1b060eb47c6f4881af5673056a4e0f5c5bf72adb7f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:46:06.123 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:05 smithi131 bash[79092]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y 2024-09-11T08:46:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQAVVuFmu1GaLRAAQazMYhSUqEl3ss44SNbUYg== 2024-09-11T08:46:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAVVuFmu1GaLRAAQazMYhSUqEl3ss44SNbUYg== == AQAVVuFmu1GaLRAAQazMYhSUqEl3ss44SNbUYg== ']' 2024-09-11T08:46:06.232 INFO:teuthology.orchestra.run.smithi131.stderr:+ sleep 5 2024-09-11T08:46:06.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:06 smithi131 podman[79106]: 2024-09-11 08:46:06.123323288 +0000 UTC m=+0.647927467 container remove 353609dc3bc565043b3c1b060eb47c6f4881af5673056a4e0f5c5bf72adb7f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-11T08:46:06.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:06 smithi131 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:46:06.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:06 smithi131 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-11T08:46:06.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:06 smithi131 systemd[1]: Stopped Ceph mgr.y for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:46:06.390 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:06 smithi131 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mgr.y.service: Consumed 44.509s CPU time. 2024-09-11T08:46:06.391 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:06 smithi131 systemd[1]: Starting Ceph mgr.y for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:46:06.810 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:06 smithi131 ceph-mon[29446]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:46:06.811 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:06 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/4282395795' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:46:06.811 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:06 smithi131 podman[79219]: 2024-09-11 08:46:06.443091038 +0000 UTC m=+0.015666541 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:46:06.811 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:06 smithi131 podman[79219]: 2024-09-11 08:46:06.582975391 +0000 UTC m=+0.155550924 container create 6b50550793feb1e09e008bebf0bba3a969ac5e675f64007402e8e5e77ddb0f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:46:06.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:06 smithi131 ceph-mon[37208]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:46:06.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:06 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/4282395795' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:46:06.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:06 smithi189 ceph-mon[32675]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:46:06.990 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:06 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/4282395795' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:46:07.140 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:06 smithi131 podman[79219]: 2024-09-11 08:46:06.81151568 +0000 UTC m=+0.384091170 container init 6b50550793feb1e09e008bebf0bba3a969ac5e675f64007402e8e5e77ddb0f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS) 2024-09-11T08:46:07.140 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:06 smithi131 podman[79219]: 2024-09-11 08:46:06.81465931 +0000 UTC m=+0.387234810 container start 6b50550793feb1e09e008bebf0bba3a969ac5e675f64007402e8e5e77ddb0f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-11T08:46:07.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:06 smithi131 ceph-mgr[79238]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 6004217919817779911 != 18374858748799134293 2024-09-11T08:46:07.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:06 smithi131 bash[79219]: 6b50550793feb1e09e008bebf0bba3a969ac5e675f64007402e8e5e77ddb0f44 2024-09-11T08:46:07.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:06 smithi131 systemd[1]: Started Ceph mgr.y for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:46:07.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:06 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:06.927+0000 7fbf1d683ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:46:07.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:06 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:06.985+0000 7fbf1d683ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:46:07.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:07.050+0000 7fbf1d683ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:46:07.568 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:07.173+0000 7fbf1d683ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:46:07.568 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:07.309+0000 7fbf1d683ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:46:07.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:07.568+0000 7fbf1d683ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:46:07.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:07.628+0000 7fbf1d683ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:46:07.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:07.789+0000 7fbf1d683ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:46:08.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:07 smithi189 ceph-mon[32675]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:46:08.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:07 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:08.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:07 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:08.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:07 smithi189 ceph-mon[32675]: Rotating authentication key for mgr.x 2024-09-11T08:46:08.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:07 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-11T08:46:08.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:07 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-11T08:46:08.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:07 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-11T08:46:08.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:07 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:46:08.241 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:07 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:46:08.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:07 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:46:08.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:07 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:46:08.242 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:07 smithi189 ceph-mon[32675]: Reconfiguring daemon mgr.x on smithi189 2024-09-11T08:46:08.390 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[37208]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:46:08.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:08.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:08.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[37208]: Rotating authentication key for mgr.x 2024-09-11T08:46:08.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-11T08:46:08.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-11T08:46:08.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-11T08:46:08.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:46:08.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:46:08.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:46:08.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:46:08.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[37208]: Reconfiguring daemon mgr.x on smithi189 2024-09-11T08:46:08.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[29446]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T08:46:08.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:08.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:08.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[29446]: Rotating authentication key for mgr.x 2024-09-11T08:46:08.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-11T08:46:08.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-11T08:46:08.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-11T08:46:08.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:46:08.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T08:46:08.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T08:46:08.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:46:08.392 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:07 smithi131 ceph-mon[29446]: Reconfiguring daemon mgr.x on smithi189 2024-09-11T08:46:08.644 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.500+0000 7f9fe4a94640 -1 mgr.server reply reply (13) Permission denied access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication 2024-09-11T08:46:08.644 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.501+0000 7f9fe228f640 -1 log_channel(cephadm) log [ERR] : Non-zero return from ['ceph', '-k', '/var/lib/ceph/mgr/ceph-x/keyring', '-n', 'mgr.x', 'tell', 'mgr.x', 'rotate-key', '-i', '-']: 2024-09-11T08:46:08.495+0000 7fc614c51640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 15390078105664244377 != 18374858748799134293 2024-09-11T08:46:08.644 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.496+0000 7fc617eb7640 1 -- 172.21.15.189:0/3128654961 >> v1:172.21.15.189:6789/0 conn(0x7fc610104e30 legacy=0x7fc610105260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:46:08.644 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.496+0000 7fc617eb7640 1 -- 172.21.15.189:0/3128654961 shutdown_connections 2024-09-11T08:46:08.644 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.496+0000 7fc617eb7640 1 -- 172.21.15.189:0/3128654961 >> 172.21.15.189:0/3128654961 conn(0x7fc61006b7c0 msgr2=0x7fc61006dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:46:08.644 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.496+0000 7fc617eb7640 1 -- 172.21.15.189:0/3128654961 shutdown_connections 2024-09-11T08:46:08.644 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.496+0000 7fc617eb7640 1 -- 172.21.15.189:0/3128654961 wait complete. 2024-09-11T08:46:08.644 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.496+0000 7fc617eb7640 1 Processor -- start 2024-09-11T08:46:08.645 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.496+0000 7fc617eb7640 1 -- start start 2024-09-11T08:46:08.645 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.496+0000 7fc617eb7640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc610115af0 con 0x7fc61010ceb0 2024-09-11T08:46:08.645 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc617eb7640 1 -- --> v1:172.21.15.189:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc610121410 con 0x7fc610104e30 2024-09-11T08:46:08.645 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc617eb7640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6101225b0 con 0x7fc610109290 2024-09-11T08:46:08.645 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc615c53640 1 --1- >> v1:172.21.15.189:6789/0 conn(0x7fc610104e30 0x7fc610115000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.189:6789/0 says I am v1:172.21.15.189:42330/0 (socket says 172.21.15.189:42330) 2024-09-11T08:46:08.645 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc615c53640 1 -- 172.21.15.189:0/617274633 learned_addr learned my addr 172.21.15.189:0/617274633 (peer_addr_for_me v1:172.21.15.189:0/0) 2024-09-11T08:46:08.645 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 <== mon.1 v1:172.21.15.189:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816286072 0 0) 0x7fc610121410 con 0x7fc610104e30 2024-09-11T08:46:08.645 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 --> v1:172.21.15.189:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5f80034e0 con 0x7fc610104e30 2024-09-11T08:46:08.645 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228211230 0 0) 0x7fc610115af0 con 0x7fc61010ceb0 2024-09-11T08:46:08.645 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc610121410 con 0x7fc61010ceb0 2024-09-11T08:46:08.645 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834557208 0 0) 0x7fc6101225b0 con 0x7fc610109290 2024-09-11T08:46:08.645 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc610115af0 con 0x7fc610109290 2024-09-11T08:46:08.645 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 <== mon.1 v1:172.21.15.189:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 748+0+0 (unknown 1375090479 0 0) 0x7fc5f80034e0 con 0x7fc610104e30 2024-09-11T08:46:08.646 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 --> v1:172.21.15.189:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc6101225b0 con 0x7fc610104e30 2024-09-11T08:46:08.646 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 <== mon.1 v1:172.21.15.189:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1834616200 0 0) 0x7fc6000017b0 con 0x7fc610104e30 2024-09-11T08:46:08.646 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 748+0+0 (unknown 2484291448 0 0) 0x7fc610121410 con 0x7fc61010ceb0 2024-09-11T08:46:08.646 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc5f80034e0 con 0x7fc61010ceb0 2024-09-11T08:46:08.646 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1834616200 0 0) 0x7fc6080018b0 con 0x7fc61010ceb0 2024-09-11T08:46:08.646 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 748+0+0 (unknown 2604173546 0 0) 0x7fc610115af0 con 0x7fc610109290 2024-09-11T08:46:08.646 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc610121410 con 0x7fc610109290 2024-09-11T08:46:08.646 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1834616200 0 0) 0x7fc6040018b0 con 0x7fc610109290 2024-09-11T08:46:08.646 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 <== mon.1 v1:172.21.15.189:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 564+0+0 (unknown 2107308269 0 0) 0x7fc6101225b0 con 0x7fc610104e30 2024-09-11T08:46:08.647 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 >> v1:172.21.15.131:6790/0 conn(0x7fc610109290 legacy=0x7fc61011b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:46:08.647 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 >> v1:172.21.15.131:6789/0 conn(0x7fc61010ceb0 legacy=0x7fc61011ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:46:08.647 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 --> v1:172.21.15.189:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc610123750 con 0x7fc610104e30 2024-09-11T08:46:08.647 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 --> v1:172.21.15.189:6789/0 -- auth(proto 2 2 bytes epoch 0) v1 -- 0x7fc610115af0 con 0x7fc610104e30 2024-09-11T08:46:08.647 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc617eb7640 1 -- 172.21.15.189:0/617274633 --> v1:172.21.15.189:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc610120460 con 0x7fc610104e30 2024-09-11T08:46:08.647 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc617eb7640 1 -- 172.21.15.189:0/617274633 --> v1:172.21.15.189:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc610120930 con 0x7fc610104e30 2024-09-11T08:46:08.647 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 <== mon.1 v1:172.21.15.189:6789/0 5 ==== config(37 keys) v1 ==== 1727+0+0 (unknown 3015570670 0 0) 0x7fc6000019d0 con 0x7fc610104e30 2024-09-11T08:46:08.647 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 <== mon.1 v1:172.21.15.189:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1834616200 0 0) 0x7fc600006e20 con 0x7fc610104e30 2024-09-11T08:46:08.647 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 <== mon.1 v1:172.21.15.189:6789/0 7 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 2850949900 0 0) 0x7fc610120930 con 0x7fc610104e30 2024-09-11T08:46:08.647 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.497+0000 7fc60e7fc640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 15390078105664244377 != 18374858748799134293 2024-09-11T08:46:08.647 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.499+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 <== mon.1 v1:172.21.15.189:6789/0 8 ==== mgrmap(e 27) v1 ==== 90804+0+0 (unknown 1421867513 0 0) 0x7fc600002290 con 0x7fc610104e30 2024-09-11T08:46:08.648 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.499+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 --> v1:172.21.15.189:6816/3807371440 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5f8073c20 con 0x7fc5f806ff60 2024-09-11T08:46:08.648 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.499+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 <== mon.1 v1:172.21.15.189:6789/0 9 ==== osd_map(64..64 src has 1..64) v4 ==== 6499+0+0 (unknown 275714938 0 0) 0x7fc60008a9e0 con 0x7fc610104e30 2024-09-11T08:46:08.648 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.500+0000 7fc60e7fc640 1 -- 172.21.15.189:0/617274633 <== mgr.24502 v1:172.21.15.189:6816/3807371440 1 ==== command_reply(tid 0: -13 access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication) v1 ==== 134+0+0 (unknown 3575743073 0 0) 0x7fc5f8073c20 con 0x7fc5f806ff60 2024-09-11T08:46:08.648 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: Error EACCES: problem getting command descriptions from mgr.x 2024-09-11T08:46:08.648 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.648 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.513+0000 7fa0076bb640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-11T08:46:08.648 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ignoring --setuser ceph since I am not root 2024-09-11T08:46:08.648 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: ignoring --setgroup ceph since I am not root 2024-09-11T08:46:08.648 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-mgr[34509]: -- 172.21.15.189:0/2765990949 <== mon.1 v1:172.21.15.189:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 268841869 0 0) 0x55ba9df84820 con 0x55ba9df90c00 2024-09-11T08:46:08.648 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-mgr[34509]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 15390078105664244377 != 18374858748799134293 2024-09-11T08:46:08.648 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.643+0000 7fb8c04c2ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:46:08.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:08 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:08.644+0000 7fbf1d683ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:46:08.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:08 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:08.709+0000 7fbf1d683ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:46:08.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.699+0000 7fb8c04c2ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:46:08.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.763+0000 7fb8c04c2ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:46:08.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:08.886+0000 7fb8c04c2ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:46:09.274 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:09.021+0000 7fb8c04c2ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:46:09.655 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:09.425+0000 7fbf1d683ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:46:09.655 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:09.536+0000 7fbf1d683ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:46:09.656 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:09.656 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:09.656 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-mon[29446]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-11T08:46:09.656 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-11T08:46:09.656 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-mon[29446]: Activating manager daemon y 2024-09-11T08:46:09.656 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-mon[29446]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:46:09.656 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-mon[29446]: from='mgr.24502 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-11T08:46:09.656 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-mon[29446]: mgrmap e28: y(active, starting, since 0.00830485s) 2024-09-11T08:46:09.656 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:09.656 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:09.656 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-mon[37208]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-11T08:46:09.657 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-11T08:46:09.657 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-mon[37208]: Activating manager daemon y 2024-09-11T08:46:09.657 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-mon[37208]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:46:09.657 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-mon[37208]: from='mgr.24502 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-11T08:46:09.657 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-mon[37208]: mgrmap e28: y(active, starting, since 0.00830485s) 2024-09-11T08:46:09.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:09 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:09.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:09 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' 2024-09-11T08:46:09.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:09 smithi189 ceph-mon[32675]: from='mgr.24502 v1:172.21.15.189:0/3245457469' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-11T08:46:09.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:09 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-11T08:46:09.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:09 smithi189 ceph-mon[32675]: Activating manager daemon y 2024-09-11T08:46:09.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:09 smithi189 ceph-mon[32675]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:46:09.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:09 smithi189 ceph-mon[32675]: from='mgr.24502 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-11T08:46:09.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:09 smithi189 ceph-mon[32675]: mgrmap e28: y(active, starting, since 0.00830485s) 2024-09-11T08:46:09.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:09.274+0000 7fb8c04c2ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:46:09.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:09.334+0000 7fb8c04c2ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:46:09.741 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:09.496+0000 7fb8c04c2ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:46:09.915 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:09.655+0000 7fbf1d683ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:46:09.916 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:09.724+0000 7fbf1d683ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:46:09.916 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:09.849+0000 7fbf1d683ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:46:09.916 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:09.915+0000 7fbf1d683ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:46:10.169 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:09 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:09.973+0000 7fbf1d683ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:46:10.169 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:10 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:10.106+0000 7fbf1d683ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:46:10.169 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:10 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:10.169+0000 7fbf1d683ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:46:10.426 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:10 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:10.234+0000 7fbf1d683ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:46:10.427 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:10 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:10.313+0000 7fbf1d683ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:46:10.427 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:10 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:10.370+0000 7fbf1d683ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:46:10.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:10 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:10.333+0000 7fb8c04c2ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:46:10.740 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:10 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:10.396+0000 7fb8c04c2ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:46:10.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:10 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:10.426+0000 7fbf1d683ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:46:11.235 INFO:teuthology.orchestra.run.smithi131.stderr:++ ceph auth get-key mgr.x 2024-09-11T08:46:11.393 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:11.098+0000 7fb8c04c2ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:46:11.394 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:11.212+0000 7fb8c04c2ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:46:11.394 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:11.335+0000 7fb8c04c2ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:46:11.394 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:11.393+0000 7fb8c04c2ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:46:11.394 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: Active manager daemon y restarted 2024-09-11T08:46:11.394 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: Activating manager daemon y 2024-09-11T08:46:11.394 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:46:11.394 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: mgrmap e29: y(active, starting, since 0.00709594s) 2024-09-11T08:46:11.394 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T08:46:11.394 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:46:11.394 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:46:11.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:46:11.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:46:11.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:46:11.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:46:11.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:46:11.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:46:11.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:46:11.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:46:11.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:46:11.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:46:11.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:46:11.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:46:11.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:46:11.395 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:46:11.396 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T08:46:11.396 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:46:11.535 INFO:teuthology.orchestra.run.smithi131.stderr:+ NK=AQDOWOFmnZrkOBAAaPOx0SCLgeliHqyvAIAv8A== 2024-09-11T08:46:11.535 INFO:teuthology.orchestra.run.smithi131.stderr:+ '[' AQAVVuFmu1GaLRAAQazMYhSUqEl3ss44SNbUYg== == AQDOWOFmnZrkOBAAaPOx0SCLgeliHqyvAIAv8A== ']' 2024-09-11T08:46:11.536 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:11.165+0000 7fbf1d683ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:46:11.536 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: [11/Sep/2024:08:46:11] ENGINE Bus STARTING 2024-09-11T08:46:11.536 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: CherryPy Checker: 2024-09-11T08:46:11.536 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: The Application mounted at '' has an empty config. 2024-09-11T08:46:11.536 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:11.536 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: [11/Sep/2024:08:46:11] ENGINE Serving on http://:::9283 2024-09-11T08:46:11.536 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: [11/Sep/2024:08:46:11] ENGINE Bus STARTED 2024-09-11T08:46:11.536 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: [11/Sep/2024:08:46:11] ENGINE Bus STOPPING 2024-09-11T08:46:11.537 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: Active manager daemon y restarted 2024-09-11T08:46:11.537 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: Activating manager daemon y 2024-09-11T08:46:11.537 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:46:11.537 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: mgrmap e29: y(active, starting, since 0.00709594s) 2024-09-11T08:46:11.537 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T08:46:11.537 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:46:11.537 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:46:11.537 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:46:11.537 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:46:11.537 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:46:11.537 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:46:11.538 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:46:11.538 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:46:11.538 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:46:11.538 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:46:11.538 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:46:11.538 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:46:11.538 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:46:11.538 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:46:11.538 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:46:11.538 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:46:11.538 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T08:46:11.538 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:46:11.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: Active manager daemon y restarted 2024-09-11T08:46:11.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: Activating manager daemon y 2024-09-11T08:46:11.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:46:11.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: mgrmap e29: y(active, starting, since 0.00709594s) 2024-09-11T08:46:11.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T08:46:11.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:46:11.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:46:11.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:46:11.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:46:11.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:46:11.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:46:11.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:46:11.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:46:11.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:46:11.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:46:11.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:46:11.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:46:11.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:46:11.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:46:11.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:46:11.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:46:11.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T08:46:11.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:46:11.643 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:11.518+0000 7fb8c04c2ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:46:11.644 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:11.584+0000 7fb8c04c2ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:46:11.644 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:11.643+0000 7fb8c04c2ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:46:11.817 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: [11/Sep/2024:08:46:11] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:46:11.817 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: [11/Sep/2024:08:46:11] ENGINE Bus STOPPED 2024-09-11T08:46:11.817 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: [11/Sep/2024:08:46:11] ENGINE Bus STARTING 2024-09-11T08:46:11.906 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:11.777+0000 7fb8c04c2ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:46:11.907 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:11.841+0000 7fb8c04c2ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:46:11.907 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:11.906+0000 7fb8c04c2ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:46:12.103 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: [11/Sep/2024:08:46:11] ENGINE Serving on http://:::9283 2024-09-11T08:46:12.103 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: [11/Sep/2024:08:46:11] ENGINE Bus STARTED 2024-09-11T08:46:12.168 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T08:46:12.177 INFO:tasks.cephadm:Teardown begin 2024-09-11T08:46:12.177 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:46:12.208 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:46:12.223 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:11.989+0000 7fb8c04c2ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:46:12.223 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:12 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:12.046+0000 7fb8c04c2ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:46:12.223 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:12 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:12.106+0000 7fb8c04c2ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:46:12.239 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T08:46:12.239 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T08:46:12.264 DEBUG:teuthology.orchestra.run.smithi189:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T08:46:12.296 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T08:46:12.296 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T08:46:12.296 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mon.a 2024-09-11T08:46:12.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-mon[29446]: from='client.? v1:172.21.15.131:0/1252429420' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:46:12.390 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-mon[29446]: Manager daemon y is now available 2024-09-11T08:46:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:46:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:46:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-mon[29446]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:46:12.391 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:12 smithi131 systemd[1]: Stopping Ceph mon.a for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:46:12.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-mon[37208]: from='client.? v1:172.21.15.131:0/1252429420' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:46:12.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-mon[37208]: Manager daemon y is now available 2024-09-11T08:46:12.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:46:12.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:12.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:46:12.391 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-mon[37208]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:46:12.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:46:12 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug there is no tcmu-runner data available 2024-09-11T08:46:12.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:12 smithi189 ceph-mon[32675]: from='client.? v1:172.21.15.131:0/1252429420' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T08:46:12.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:12 smithi189 ceph-mon[32675]: Manager daemon y is now available 2024-09-11T08:46:12.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:12 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:46:12.490 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:12 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:46:12.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:12 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:46:12.491 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:12 smithi189 ceph-mon[32675]: from='mgr.25361 v1:172.21.15.131:0/127561844' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:46:12.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: [11/Sep/2024:08:46:12] ENGINE Bus STOPPING 2024-09-11T08:46:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a[29442]: 2024-09-11T08:46:12.628+0000 7f2570cb5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:46:12.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a[29442]: 2024-09-11T08:46:12.628+0000 7f2570cb5640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T08:46:13.013 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:12 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:12.907+0000 7fb8c04c2ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:46:13.013 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:12 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: [11/Sep/2024:08:46:12] ENGINE Bus STARTING 2024-09-11T08:46:13.013 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:12 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: CherryPy Checker: 2024-09-11T08:46:13.013 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:12 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: The Application mounted at '' has an empty config. 2024-09-11T08:46:13.013 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:12 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: 2024-09-11T08:46:13.013 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:13 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: [11/Sep/2024:08:46:13] ENGINE Serving on http://:::9283 2024-09-11T08:46:13.013 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:13 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x[34505]: [11/Sep/2024:08:46:13] ENGINE Bus STARTED 2024-09-11T08:46:13.204 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: [11/Sep/2024:08:46:12] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:46:13.205 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: [11/Sep/2024:08:46:12] ENGINE Bus STOPPED 2024-09-11T08:46:13.205 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: [11/Sep/2024:08:46:12] ENGINE Bus STARTING 2024-09-11T08:46:13.205 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:12.983+0000 7fbec01c4640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:46:12] ENGINE Error in HTTPServer.serve 2024-09-11T08:46:13.205 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: Traceback (most recent call last): 2024-09-11T08:46:13.205 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:46:13.205 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: self._connections.run(self.expiration_interval) 2024-09-11T08:46:13.205 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:46:13.205 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: self._run(expiration_interval) 2024-09-11T08:46:13.205 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:46:13.205 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:46:13.205 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:46:13.206 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:46:13.206 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:46:13.206 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: s = self.context.wrap_socket( 2024-09-11T08:46:13.206 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:46:13.206 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: return self.sslsocket_class._create( 2024-09-11T08:46:13.206 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:46:13.206 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: self.do_handshake() 2024-09-11T08:46:13.206 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:46:13.206 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: self._sslobj.do_handshake() 2024-09-11T08:46:13.206 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:46:13.206 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:12 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:13.206 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:13 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: [11/Sep/2024:08:46:13] ENGINE Serving on http://:::9283 2024-09-11T08:46:13.206 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:13 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: [11/Sep/2024:08:46:13] ENGINE Bus STARTED 2024-09-11T08:46:13.640 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:13 smithi131 podman[79681]: 2024-09-11 08:46:13.432234514 +0000 UTC m=+0.909994288 container died 4f1b488f6614c3694ce35cf32141520f4df7d38fdd7a811583c46cf618e61859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T08:46:13.984 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:13 smithi131 podman[79681]: 2024-09-11 08:46:13.783583111 +0000 UTC m=+1.261342917 container cleanup 4f1b488f6614c3694ce35cf32141520f4df7d38fdd7a811583c46cf618e61859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-11T08:46:13.984 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:13 smithi131 bash[79681]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a 2024-09-11T08:46:14.228 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T08:46:14.251 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:13 smithi131 podman[79752]: 2024-09-11 08:46:13.98517124 +0000 UTC m=+1.179509599 container remove 4f1b488f6614c3694ce35cf32141520f4df7d38fdd7a811583c46cf618e61859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T08:46:14.251 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:14 smithi131 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T08:46:14.251 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:14 smithi131 systemd[1]: Stopped Ceph mon.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:46:14.251 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 11 08:46:14 smithi131 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mon.a.service: Consumed 13.492s CPU time. 2024-09-11T08:46:14.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:46:14.285 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T08:46:14.285 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-11T08:46:14.285 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mon.c 2024-09-11T08:46:14.588 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:14 smithi131 systemd[1]: Stopping Ceph mon.c for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:46:15.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:15 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-c[37202]: 2024-09-11T08:46:15.183+0000 7fb643926640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:46:15.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:15 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-c[37202]: 2024-09-11T08:46:15.183+0000 7fb643926640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T08:46:15.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:15 smithi131 podman[80086]: 2024-09-11 08:46:15.543170816 +0000 UTC m=+0.783225671 container died 569d21b6765f851a50830410f022e15ec8da7ff7ab30888d1f4577303fc07168 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:46:16.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:15 smithi131 podman[80086]: 2024-09-11 08:46:15.919633626 +0000 UTC m=+1.159688435 container cleanup 569d21b6765f851a50830410f022e15ec8da7ff7ab30888d1f4577303fc07168 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-c, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T08:46:16.140 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:15 smithi131 bash[80086]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-c 2024-09-11T08:46:16.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 11 08:46:16 smithi131 podman[80153]: 2024-09-11 08:46:16.196451734 +0000 UTC m=+0.759691491 container remove 569d21b6765f851a50830410f022e15ec8da7ff7ab30888d1f4577303fc07168 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T08:46:16.595 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T08:46:16.633 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:46:16.633 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-11T08:46:16.633 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-11T08:46:16.633 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mon.b 2024-09-11T08:46:16.932 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:16 smithi189 systemd[1]: Stopping Ceph mon.b for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:46:17.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:16 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-b[32658]: 2024-09-11T08:46:16.932+0000 7f8dffe30640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:46:17.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:16 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-b[32658]: 2024-09-11T08:46:16.932+0000 7f8dffe30640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T08:46:17.240 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:17 smithi189 podman[69793]: 2024-09-11 08:46:17.064399267 +0000 UTC m=+0.233866976 container died f858a635fc72da4510d87b28c757ec6bb7887c4ae42f4d72dd26197ec7f45a1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:46:17.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:17 smithi189 podman[69793]: 2024-09-11 08:46:17.415428744 +0000 UTC m=+0.584896446 container cleanup f858a635fc72da4510d87b28c757ec6bb7887c4ae42f4d72dd26197ec7f45a1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-11T08:46:17.740 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 11 08:46:17 smithi189 bash[69793]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mon-b 2024-09-11T08:46:17.994 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T08:46:18.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:46:18.039 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-11T08:46:18.040 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T08:46:18.040 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mgr.y 2024-09-11T08:46:18.334 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:18 smithi131 systemd[1]: Stopping Ceph mgr.y for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:46:18.593 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:18 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y[79234]: 2024-09-11T08:46:18.333+0000 7fbf19618640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:46:18.593 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:18 smithi131 podman[80274]: 2024-09-11 08:46:18.365224382 +0000 UTC m=+0.138271358 container died 6b50550793feb1e09e008bebf0bba3a969ac5e675f64007402e8e5e77ddb0f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T08:46:18.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:18 smithi131 podman[80274]: 2024-09-11 08:46:18.701389305 +0000 UTC m=+0.474436279 container cleanup 6b50550793feb1e09e008bebf0bba3a969ac5e675f64007402e8e5e77ddb0f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=) 2024-09-11T08:46:18.890 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 11 08:46:18 smithi131 bash[80274]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-y 2024-09-11T08:46:19.030 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T08:46:19.066 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:46:19.067 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T08:46:19.067 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T08:46:19.067 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mgr.x 2024-09-11T08:46:19.390 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:19 smithi189 systemd[1]: Stopping Ceph mgr.x for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:46:19.669 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:19 smithi189 podman[69915]: 2024-09-11 08:46:19.390537508 +0000 UTC m=+0.127877670 container died 9a2cf3ca6884231bfca591d5e8604c6d0268882b2a61fa8903741ff46a530895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T08:46:19.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:19 smithi189 podman[69915]: 2024-09-11 08:46:19.744654215 +0000 UTC m=+0.481994378 container cleanup 9a2cf3ca6884231bfca591d5e8604c6d0268882b2a61fa8903741ff46a530895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:46:19.990 INFO:journalctl@ceph.mgr.x.smithi189.stdout:Sep 11 08:46:19 smithi189 bash[69915]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-mgr-x 2024-09-11T08:46:20.177 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T08:46:20.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:46:20.214 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T08:46:20.214 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T08:46:20.214 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.0 2024-09-11T08:46:20.530 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:46:20 smithi131 systemd[1]: Stopping Ceph osd.0 for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:46:20.891 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:46:20 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-0[41071]: 2024-09-11T08:46:20.530+0000 7f123d10d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:46:20.891 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:46:20 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-0[41071]: 2024-09-11T08:46:20.530+0000 7f123d10d640 -1 osd.0 66 *** Got signal Terminated *** 2024-09-11T08:46:20.891 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:46:20 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-0[41071]: 2024-09-11T08:46:20.530+0000 7f123d10d640 -1 osd.0 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:46:26.289 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:46:26 smithi131 podman[80396]: 2024-09-11 08:46:26.019728746 +0000 UTC m=+5.613895399 container died 7de38a9c5d1f33fcbb3089dd0dd21f9d34f3d3853aea15e231ef2bc6f7cf7376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-11T08:46:26.639 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:46:26 smithi131 podman[80396]: 2024-09-11 08:46:26.380768224 +0000 UTC m=+5.974934865 container cleanup 7de38a9c5d1f33fcbb3089dd0dd21f9d34f3d3853aea15e231ef2bc6f7cf7376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:46:26.640 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:46:26 smithi131 bash[80396]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-0 2024-09-11T08:46:26.890 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:46:26 smithi131 podman[80412]: 2024-09-11 08:46:26.640575049 +0000 UTC m=+0.621198629 container remove 7de38a9c5d1f33fcbb3089dd0dd21f9d34f3d3853aea15e231ef2bc6f7cf7376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:46:27.205 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:46:26 smithi131 podman[80505]: 2024-09-11 08:46:26.856574309 +0000 UTC m=+0.015726890 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:46:27.205 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:46:26 smithi131 podman[80505]: 2024-09-11 08:46:26.975066311 +0000 UTC m=+0.134218936 container create b5ca48b43164b684b93c7feb326c214f995621a62a37031554761d04e056a05a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T08:46:27.205 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:46:27 smithi131 podman[80505]: 2024-09-11 08:46:27.20285331 +0000 UTC m=+0.362005893 container init b5ca48b43164b684b93c7feb326c214f995621a62a37031554761d04e056a05a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:46:27.205 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:46:27 smithi131 podman[80505]: 2024-09-11 08:46:27.206144142 +0000 UTC m=+0.365296722 container start b5ca48b43164b684b93c7feb326c214f995621a62a37031554761d04e056a05a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T08:46:27.640 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:46:27 smithi131 podman[80505]: 2024-09-11 08:46:27.303047959 +0000 UTC m=+0.462200539 container attach b5ca48b43164b684b93c7feb326c214f995621a62a37031554761d04e056a05a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:46:27.641 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 11 08:46:27 smithi131 podman[80505]: 2024-09-11 08:46:27.380173019 +0000 UTC m=+0.539325599 container died b5ca48b43164b684b93c7feb326c214f995621a62a37031554761d04e056a05a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:46:27.942 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T08:46:27.980 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:46:27.981 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T08:46:27.981 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T08:46:27.981 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.1 2024-09-11T08:46:28.309 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:46:28 smithi131 systemd[1]: Stopping Ceph osd.1 for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:46:28.640 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:46:28 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-1[46640]: 2024-09-11T08:46:28.309+0000 7f34ebaae640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:46:28.640 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:46:28 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-1[46640]: 2024-09-11T08:46:28.309+0000 7f34ebaae640 -1 osd.1 66 *** Got signal Terminated *** 2024-09-11T08:46:28.640 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:46:28 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-1[46640]: 2024-09-11T08:46:28.309+0000 7f34ebaae640 -1 osd.1 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:46:33.893 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:46:33 smithi131 podman[80608]: 2024-09-11 08:46:33.634993612 +0000 UTC m=+5.430298592 container died 6ac71861ac266c86c2c7b5a1c99ac7d464dbe544f580c9abd103e4c6f9be4e71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-09-11T08:46:34.268 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:46:34 smithi131 podman[80608]: 2024-09-11 08:46:34.001386468 +0000 UTC m=+5.796691451 container cleanup 6ac71861ac266c86c2c7b5a1c99ac7d464dbe544f580c9abd103e4c6f9be4e71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:46:34.269 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:46:34 smithi131 bash[80608]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-1 2024-09-11T08:46:34.612 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:46:34 smithi131 podman[80624]: 2024-09-11 08:46:34.269431191 +0000 UTC m=+0.624975834 container remove 6ac71861ac266c86c2c7b5a1c99ac7d464dbe544f580c9abd103e4c6f9be4e71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T08:46:34.889 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:46:34 smithi131 podman[80708]: 2024-09-11 08:46:34.516152306 +0000 UTC m=+0.015946704 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:46:34.889 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:46:34 smithi131 podman[80708]: 2024-09-11 08:46:34.620414141 +0000 UTC m=+0.120208581 container create cfd0ee30714821fdbabb7b2d4bb0012479de3af272798709436624227d1a8c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-11T08:46:34.889 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:46:34 smithi131 podman[80708]: 2024-09-11 08:46:34.81468515 +0000 UTC m=+0.314479547 container init cfd0ee30714821fdbabb7b2d4bb0012479de3af272798709436624227d1a8c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:46:34.889 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:46:34 smithi131 podman[80708]: 2024-09-11 08:46:34.818012855 +0000 UTC m=+0.317807255 container start cfd0ee30714821fdbabb7b2d4bb0012479de3af272798709436624227d1a8c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-11T08:46:34.889 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:46:34 smithi131 podman[80708]: 2024-09-11 08:46:34.889742577 +0000 UTC m=+0.389536984 container attach cfd0ee30714821fdbabb7b2d4bb0012479de3af272798709436624227d1a8c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-11T08:46:35.141 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 11 08:46:34 smithi131 podman[80708]: 2024-09-11 08:46:34.994142266 +0000 UTC m=+0.493936664 container died cfd0ee30714821fdbabb7b2d4bb0012479de3af272798709436624227d1a8c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T08:46:35.616 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T08:46:35.659 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:46:35.660 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T08:46:35.660 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T08:46:35.660 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.2 2024-09-11T08:46:35.890 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:46:35 smithi131 systemd[1]: Stopping Ceph osd.2 for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:46:36.257 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:46:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-2[52627]: 2024-09-11T08:46:36.004+0000 7f7dc8993640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:46:36.257 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:46:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-2[52627]: 2024-09-11T08:46:36.004+0000 7f7dc8993640 -1 osd.2 66 *** Got signal Terminated *** 2024-09-11T08:46:36.257 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:46:36 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-2[52627]: 2024-09-11T08:46:36.004+0000 7f7dc8993640 -1 osd.2 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:46:41.605 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:46:41 smithi131 podman[80821]: 2024-09-11 08:46:41.32795639 +0000 UTC m=+5.424854471 container died ef192887c825840ea9011f82ffda076645a48cce4c4210c388f444e35e87c226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:46:41.891 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:46:41 smithi131 podman[80821]: 2024-09-11 08:46:41.605353262 +0000 UTC m=+5.702251381 container cleanup ef192887c825840ea9011f82ffda076645a48cce4c4210c388f444e35e87c226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T08:46:41.891 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:46:41 smithi131 bash[80821]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-2 2024-09-11T08:46:41.891 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:46:41 smithi131 podman[80837]: 2024-09-11 08:46:41.848329324 +0000 UTC m=+0.523660553 container remove ef192887c825840ea9011f82ffda076645a48cce4c4210c388f444e35e87c226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:46:42.459 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:46:42 smithi131 podman[80926]: 2024-09-11 08:46:42.065895647 +0000 UTC m=+0.015772928 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:46:42.459 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:46:42 smithi131 podman[80926]: 2024-09-11 08:46:42.16589082 +0000 UTC m=+0.115768096 container create 1cae2c62480525666ace7bc0b6dcb4f7f3332cfe011f963d8fe575899d68b69c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-11T08:46:42.459 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:46:42 smithi131 podman[80926]: 2024-09-11 08:46:42.376589336 +0000 UTC m=+0.326466612 container init 1cae2c62480525666ace7bc0b6dcb4f7f3332cfe011f963d8fe575899d68b69c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:46:42.459 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:46:42 smithi131 podman[80926]: 2024-09-11 08:46:42.379876384 +0000 UTC m=+0.329753664 container start 1cae2c62480525666ace7bc0b6dcb4f7f3332cfe011f963d8fe575899d68b69c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:46:42.459 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:46:42 smithi131 podman[80926]: 2024-09-11 08:46:42.459963842 +0000 UTC m=+0.409841119 container attach 1cae2c62480525666ace7bc0b6dcb4f7f3332cfe011f963d8fe575899d68b69c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:46:42.886 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 11 08:46:42 smithi131 podman[80926]: 2024-09-11 08:46:42.55365879 +0000 UTC m=+0.503536068 container died 1cae2c62480525666ace7bc0b6dcb4f7f3332cfe011f963d8fe575899d68b69c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-09-11T08:46:43.183 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T08:46:43.223 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:46:43.224 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T08:46:43.224 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T08:46:43.224 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.3 2024-09-11T08:46:43.566 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:46:43 smithi131 systemd[1]: Stopping Ceph osd.3 for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:46:43.893 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:46:43 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-3[58709]: 2024-09-11T08:46:43.566+0000 7f095bba3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:46:43.893 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:46:43 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-3[58709]: 2024-09-11T08:46:43.566+0000 7f095bba3640 -1 osd.3 66 *** Got signal Terminated *** 2024-09-11T08:46:43.893 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:46:43 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-3[58709]: 2024-09-11T08:46:43.566+0000 7f095bba3640 -1 osd.3 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:46:47.490 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:46:47 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:46:47.231+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:46:48.490 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:46:48 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:46:48.209+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:46:48.891 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:46:48 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-3[58709]: 2024-09-11T08:46:48.548+0000 7f09551b5640 -1 osd.3 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:25.001311+0000 front 2024-09-11T08:46:25.001333+0000 (oldest deadline 2024-09-11T08:46:48.501395+0000) 2024-09-11T08:46:48.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:46:48 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:46:48.634+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:46:49.158 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:46:48 smithi131 podman[81028]: 2024-09-11 08:46:48.89995048 +0000 UTC m=+5.455853640 container died d8f84f5f9298158baf9b9185aa730c623f5c11aac35de5509cfabced2e3bf095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T08:46:49.490 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:46:49 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:46:49.173+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:46:49.542 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:46:49 smithi131 podman[81028]: 2024-09-11 08:46:49.258135246 +0000 UTC m=+5.814038397 container cleanup d8f84f5f9298158baf9b9185aa730c623f5c11aac35de5509cfabced2e3bf095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-11T08:46:49.542 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:46:49 smithi131 bash[81028]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-3 2024-09-11T08:46:49.849 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:46:49 smithi131 podman[81043]: 2024-09-11 08:46:49.542643184 +0000 UTC m=+0.636309328 container remove d8f84f5f9298158baf9b9185aa730c623f5c11aac35de5509cfabced2e3bf095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T08:46:49.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:46:49 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:46:49.587+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:46:50.141 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:46:49 smithi131 podman[81129]: 2024-09-11 08:46:49.753341502 +0000 UTC m=+0.015871147 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:46:50.141 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:46:49 smithi131 podman[81129]: 2024-09-11 08:46:49.868687304 +0000 UTC m=+0.131216989 container create 223103b42add2c657b8d5a7a58ceeb8d228af6497336e365027eb369bd73be43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T08:46:50.141 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:46:50 smithi131 podman[81129]: 2024-09-11 08:46:50.079616549 +0000 UTC m=+0.342146185 container init 223103b42add2c657b8d5a7a58ceeb8d228af6497336e365027eb369bd73be43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T08:46:50.141 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:46:50 smithi131 podman[81129]: 2024-09-11 08:46:50.082787207 +0000 UTC m=+0.345316852 container start 223103b42add2c657b8d5a7a58ceeb8d228af6497336e365027eb369bd73be43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-11T08:46:50.490 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:46:50 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:46:50.129+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:46:50.615 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:46:50 smithi131 podman[81129]: 2024-09-11 08:46:50.171281946 +0000 UTC m=+0.433811592 container attach 223103b42add2c657b8d5a7a58ceeb8d228af6497336e365027eb369bd73be43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T08:46:50.616 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 11 08:46:50 smithi131 podman[81129]: 2024-09-11 08:46:50.257017108 +0000 UTC m=+0.519546745 container died 223103b42add2c657b8d5a7a58ceeb8d228af6497336e365027eb369bd73be43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T08:46:50.922 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T08:46:50.961 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:46:50.961 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T08:46:50.962 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T08:46:50.962 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.4 2024-09-11T08:46:50.990 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:50 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4[37434]: 2024-09-11T08:46:50.707+0000 7f59dd613640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:25.169012+0000 front 2024-09-11T08:46:25.168790+0000 (oldest deadline 2024-09-11T08:46:49.868867+0000) 2024-09-11T08:46:50.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:46:50 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:46:50.591+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:46:51.296 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:46:51 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:46:51.178+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:46:51.296 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:51 smithi189 systemd[1]: Stopping Ceph osd.4 for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:46:51.296 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:51 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4[37434]: 2024-09-11T08:46:51.294+0000 7f59e1ffd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:46:51.643 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:51 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4[37434]: 2024-09-11T08:46:51.294+0000 7f59e1ffd640 -1 osd.4 66 *** Got signal Terminated *** 2024-09-11T08:46:51.643 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:51 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4[37434]: 2024-09-11T08:46:51.294+0000 7f59e1ffd640 -1 osd.4 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:46:51.990 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:51 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4[37434]: 2024-09-11T08:46:51.688+0000 7f59dd613640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:25.169012+0000 front 2024-09-11T08:46:25.168790+0000 (oldest deadline 2024-09-11T08:46:49.868867+0000) 2024-09-11T08:46:51.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:46:51 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:46:51.642+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:46:52.490 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:46:52 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:46:52.202+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:46:52.990 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:52 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4[37434]: 2024-09-11T08:46:52.733+0000 7f59dd613640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:25.169012+0000 front 2024-09-11T08:46:25.168790+0000 (oldest deadline 2024-09-11T08:46:49.868867+0000) 2024-09-11T08:46:52.990 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:46:52 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:46:52.550+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.609219+0000 front 2024-09-11T08:46:21.609364+0000 (oldest deadline 2024-09-11T08:46:51.609930+0000) 2024-09-11T08:46:52.991 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:46:52 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:46:52.664+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:46:53.492 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:46:53 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:46:53.197+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:46:53.990 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:53 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4[37434]: 2024-09-11T08:46:53.733+0000 7f59dd613640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:25.169012+0000 front 2024-09-11T08:46:25.168790+0000 (oldest deadline 2024-09-11T08:46:49.868867+0000) 2024-09-11T08:46:53.990 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:46:53 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:46:53.550+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.609219+0000 front 2024-09-11T08:46:21.609364+0000 (oldest deadline 2024-09-11T08:46:51.609930+0000) 2024-09-11T08:46:53.991 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:46:53 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:46:53.618+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:46:54.489 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:46:54 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:46:54.187+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:46:54.990 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:54 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4[37434]: 2024-09-11T08:46:54.683+0000 7f59dd613640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:25.169012+0000 front 2024-09-11T08:46:25.168790+0000 (oldest deadline 2024-09-11T08:46:49.868867+0000) 2024-09-11T08:46:54.990 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:46:54 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:46:54.520+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.609219+0000 front 2024-09-11T08:46:21.609364+0000 (oldest deadline 2024-09-11T08:46:51.609930+0000) 2024-09-11T08:46:54.991 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:46:54 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:46:54.659+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:46:55.490 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:46:55 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:46:55.226+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:46:55.990 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:55 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4[37434]: 2024-09-11T08:46:55.664+0000 7f59dd613640 -1 osd.4 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:25.169012+0000 front 2024-09-11T08:46:25.168790+0000 (oldest deadline 2024-09-11T08:46:49.868867+0000) 2024-09-11T08:46:55.991 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:46:55 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:46:55.516+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.609219+0000 front 2024-09-11T08:46:21.609364+0000 (oldest deadline 2024-09-11T08:46:51.609930+0000) 2024-09-11T08:46:55.991 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:46:55 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:46:55.612+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:46:56.490 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:46:56 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:46:56.194+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:46:56.923 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:56 smithi189 podman[70052]: 2024-09-11 08:46:56.656359468 +0000 UTC m=+5.496324296 container died 51274f807d269131a5f136c180c39a99a35378b7e185a9941321dd1a79abe01e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-11T08:46:56.923 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:46:56 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:46:56.518+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.609219+0000 front 2024-09-11T08:46:21.609364+0000 (oldest deadline 2024-09-11T08:46:51.609930+0000) 2024-09-11T08:46:56.923 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:46:56 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:46:56.659+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:46:56.923 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:46:56 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:46:56.659+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:46:57.237 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:57 smithi189 podman[70052]: 2024-09-11 08:46:57.012524278 +0000 UTC m=+5.852489099 container cleanup 51274f807d269131a5f136c180c39a99a35378b7e185a9941321dd1a79abe01e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T08:46:57.238 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:57 smithi189 bash[70052]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4 2024-09-11T08:46:57.490 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:57 smithi189 podman[70066]: 2024-09-11 08:46:57.280887489 +0000 UTC m=+0.621363939 container remove 51274f807d269131a5f136c180c39a99a35378b7e185a9941321dd1a79abe01e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T08:46:57.490 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:46:57 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:46:57.237+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:46:57.490 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:46:57 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:46:57.237+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:30.627617+0000 front 2024-09-11T08:46:30.627656+0000 (oldest deadline 2024-09-11T08:46:56.527588+0000) 2024-09-11T08:46:57.842 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:46:57 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:46:57.533+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.609219+0000 front 2024-09-11T08:46:21.609364+0000 (oldest deadline 2024-09-11T08:46:51.609930+0000) 2024-09-11T08:46:57.842 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:46:57 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:46:57.632+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:46:57.842 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:46:57 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:46:57.632+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:46:57.842 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:57 smithi189 podman[70158]: 2024-09-11 08:46:57.53924665 +0000 UTC m=+0.015564399 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:46:57.842 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:57 smithi189 podman[70158]: 2024-09-11 08:46:57.640075849 +0000 UTC m=+0.116393593 container create 6cba8e3c8b83f419a39c1c09d1939a06e783110edf413c330ec4ef8547bbf042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-11T08:46:58.240 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:57 smithi189 podman[70158]: 2024-09-11 08:46:57.842404742 +0000 UTC m=+0.318722478 container init 6cba8e3c8b83f419a39c1c09d1939a06e783110edf413c330ec4ef8547bbf042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T08:46:58.240 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:57 smithi189 podman[70158]: 2024-09-11 08:46:57.845772952 +0000 UTC m=+0.322090698 container start 6cba8e3c8b83f419a39c1c09d1939a06e783110edf413c330ec4ef8547bbf042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-09-11T08:46:58.240 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:57 smithi189 podman[70158]: 2024-09-11 08:46:57.917455871 +0000 UTC m=+0.393773614 container attach 6cba8e3c8b83f419a39c1c09d1939a06e783110edf413c330ec4ef8547bbf042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T08:46:58.240 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 11 08:46:58 smithi189 podman[70158]: 2024-09-11 08:46:58.021373058 +0000 UTC m=+0.497690805 container died 6cba8e3c8b83f419a39c1c09d1939a06e783110edf413c330ec4ef8547bbf042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-11T08:46:58.603 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:46:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:46:58.494+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.609219+0000 front 2024-09-11T08:46:21.609364+0000 (oldest deadline 2024-09-11T08:46:51.609930+0000) 2024-09-11T08:46:58.604 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:46:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:46:58.260+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:46:58.604 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:46:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:46:58.260+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:30.627617+0000 front 2024-09-11T08:46:30.627656+0000 (oldest deadline 2024-09-11T08:46:56.527588+0000) 2024-09-11T08:46:58.624 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T08:46:58.705 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:46:58.705 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T08:46:58.705 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T08:46:58.706 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.5 2024-09-11T08:46:58.990 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:46:58 smithi189 systemd[1]: Stopping Ceph osd.5 for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:46:58.991 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:46:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:46:58.639+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:46:58.991 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:46:58 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:46:58.639+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:46:59.490 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:46:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:46:59.257+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:46:59.490 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:46:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:46:59.257+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:30.627617+0000 front 2024-09-11T08:46:30.627656+0000 (oldest deadline 2024-09-11T08:46:56.527588+0000) 2024-09-11T08:46:59.490 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:46:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:46:59.032+0000 7f6d44205640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:46:59.490 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:46:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:46:59.032+0000 7f6d44205640 -1 osd.5 66 *** Got signal Terminated *** 2024-09-11T08:46:59.490 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:46:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:46:59.032+0000 7f6d44205640 -1 osd.5 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:46:59.491 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:46:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:46:59.489+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.609219+0000 front 2024-09-11T08:46:21.609364+0000 (oldest deadline 2024-09-11T08:46:51.609930+0000) 2024-09-11T08:46:59.740 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:46:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:46:59.489+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:32.710898+0000 front 2024-09-11T08:46:32.710839+0000 (oldest deadline 2024-09-11T08:46:58.610704+0000) 2024-09-11T08:46:59.740 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:46:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:46:59.612+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:46:59.740 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:46:59 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:46:59.612+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:00.572 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:00 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:47:00.462+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.609219+0000 front 2024-09-11T08:46:21.609364+0000 (oldest deadline 2024-09-11T08:46:51.609930+0000) 2024-09-11T08:47:00.572 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:00 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:47:00.462+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:32.710898+0000 front 2024-09-11T08:46:32.710839+0000 (oldest deadline 2024-09-11T08:46:58.610704+0000) 2024-09-11T08:47:00.572 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:00 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:00.301+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:47:00.572 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:00 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:00.301+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:30.627617+0000 front 2024-09-11T08:46:30.627656+0000 (oldest deadline 2024-09-11T08:46:56.527588+0000) 2024-09-11T08:47:00.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:00 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:00.571+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:00.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:00 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:00.571+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:01.593 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:47:01.476+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.609219+0000 front 2024-09-11T08:46:21.609364+0000 (oldest deadline 2024-09-11T08:46:51.609930+0000) 2024-09-11T08:47:01.593 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:47:01.476+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:32.710898+0000 front 2024-09-11T08:46:32.710839+0000 (oldest deadline 2024-09-11T08:46:58.610704+0000) 2024-09-11T08:47:01.593 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:01.269+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:47:01.593 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:01.269+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:30.627617+0000 front 2024-09-11T08:46:30.627656+0000 (oldest deadline 2024-09-11T08:46:56.527588+0000) 2024-09-11T08:47:01.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:01.592+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:01.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:01 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:01.592+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:02.596 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:02 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:47:02.487+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.609219+0000 front 2024-09-11T08:46:21.609364+0000 (oldest deadline 2024-09-11T08:46:51.609930+0000) 2024-09-11T08:47:02.596 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:02 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:47:02.487+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:32.710898+0000 front 2024-09-11T08:46:32.710839+0000 (oldest deadline 2024-09-11T08:46:58.610704+0000) 2024-09-11T08:47:02.596 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:02 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:02.264+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:47:02.597 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:02 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:02.264+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:30.627617+0000 front 2024-09-11T08:46:30.627656+0000 (oldest deadline 2024-09-11T08:46:56.527588+0000) 2024-09-11T08:47:02.597 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:02 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:02.264+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:40.529169+0000 front 2024-09-11T08:46:40.529077+0000 (oldest deadline 2024-09-11T08:47:01.628993+0000) 2024-09-11T08:47:02.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:02 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:02.596+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:02.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:02 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:02.596+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:02.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:02 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:02.596+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:03.639 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:03 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:47:03.483+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.609219+0000 front 2024-09-11T08:46:21.609364+0000 (oldest deadline 2024-09-11T08:46:51.609930+0000) 2024-09-11T08:47:03.639 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:03 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5[43254]: 2024-09-11T08:47:03.483+0000 7f6d42020640 -1 osd.5 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:32.710898+0000 front 2024-09-11T08:46:32.710839+0000 (oldest deadline 2024-09-11T08:46:58.610704+0000) 2024-09-11T08:47:03.639 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:03 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:03.270+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:47:03.639 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:03 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:03.270+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:30.627617+0000 front 2024-09-11T08:46:30.627656+0000 (oldest deadline 2024-09-11T08:46:56.527588+0000) 2024-09-11T08:47:03.639 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:03 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:03.270+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:40.529169+0000 front 2024-09-11T08:46:40.529077+0000 (oldest deadline 2024-09-11T08:47:01.628993+0000) 2024-09-11T08:47:03.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:03 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:03.638+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:03.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:03 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:03.638+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:03.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:03 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:03.638+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:04.644 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:04 smithi189 podman[70261]: 2024-09-11 08:47:04.43328745 +0000 UTC m=+5.507680064 container died 56ef04facab25db0af282c744a14634e2257b6f579dd658625bcdfd36fe2e15a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:47:04.644 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:04 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:04.307+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:47:04.644 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:04 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:04.307+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:30.627617+0000 front 2024-09-11T08:46:30.627656+0000 (oldest deadline 2024-09-11T08:46:56.527588+0000) 2024-09-11T08:47:04.644 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:04 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:04.307+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:40.529169+0000 front 2024-09-11T08:46:40.529077+0000 (oldest deadline 2024-09-11T08:47:01.628993+0000) 2024-09-11T08:47:04.957 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:04 smithi189 podman[70261]: 2024-09-11 08:47:04.748662053 +0000 UTC m=+5.823054649 container cleanup 56ef04facab25db0af282c744a14634e2257b6f579dd658625bcdfd36fe2e15a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T08:47:04.957 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:04 smithi189 bash[70261]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5 2024-09-11T08:47:04.958 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:04 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:04.644+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:04.958 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:04 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:04.644+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:04.958 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:04 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:04.644+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:05.240 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:04 smithi189 podman[70276]: 2024-09-11 08:47:04.958224355 +0000 UTC m=+0.520529849 container remove 56ef04facab25db0af282c744a14634e2257b6f579dd658625bcdfd36fe2e15a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:47:05.636 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:05 smithi189 podman[70365]: 2024-09-11 08:47:05.275634626 +0000 UTC m=+0.015964673 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:47:05.636 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:05 smithi189 podman[70365]: 2024-09-11 08:47:05.425753158 +0000 UTC m=+0.166083238 container create 83fb727bd822180713c546be34fafa104430d672e5b47066d0867b8badb403c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T08:47:05.636 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:05 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:05.314+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:47:05.636 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:05 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:05.314+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:30.627617+0000 front 2024-09-11T08:46:30.627656+0000 (oldest deadline 2024-09-11T08:46:56.527588+0000) 2024-09-11T08:47:05.636 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:05 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:05.314+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:40.529169+0000 front 2024-09-11T08:46:40.529077+0000 (oldest deadline 2024-09-11T08:47:01.628993+0000) 2024-09-11T08:47:05.991 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:05 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:05.690+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:05.991 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:05 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:05.690+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:05.991 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:05 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:05.690+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:05.991 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:05 smithi189 podman[70365]: 2024-09-11 08:47:05.636558193 +0000 UTC m=+0.376888238 container init 83fb727bd822180713c546be34fafa104430d672e5b47066d0867b8badb403c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T08:47:05.991 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:05 smithi189 podman[70365]: 2024-09-11 08:47:05.640036554 +0000 UTC m=+0.380366598 container start 83fb727bd822180713c546be34fafa104430d672e5b47066d0867b8badb403c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0) 2024-09-11T08:47:05.991 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:05 smithi189 podman[70365]: 2024-09-11 08:47:05.728151151 +0000 UTC m=+0.468481189 container attach 83fb727bd822180713c546be34fafa104430d672e5b47066d0867b8badb403c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T08:47:05.991 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 11 08:47:05 smithi189 podman[70365]: 2024-09-11 08:47:05.813525493 +0000 UTC m=+0.553855529 container died 83fb727bd822180713c546be34fafa104430d672e5b47066d0867b8badb403c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:47:06.376 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T08:47:06.417 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:47:06.417 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T08:47:06.417 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T08:47:06.418 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.6 2024-09-11T08:47:06.709 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:06 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:06.362+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:47:06.709 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:06 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:06.362+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:30.627617+0000 front 2024-09-11T08:46:30.627656+0000 (oldest deadline 2024-09-11T08:46:56.527588+0000) 2024-09-11T08:47:06.709 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:06 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:06.362+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:40.529169+0000 front 2024-09-11T08:46:40.529077+0000 (oldest deadline 2024-09-11T08:47:01.628993+0000) 2024-09-11T08:47:06.709 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:06 smithi189 systemd[1]: Stopping Ceph osd.6 for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:47:06.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:06 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:06.708+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:06.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:06 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:06.708+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:06.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:06 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:06.708+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:06.990 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:06 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:06.753+0000 7f6d2722a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:47:06.990 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:06 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:06.753+0000 7f6d2722a640 -1 osd.6 66 *** Got signal Terminated *** 2024-09-11T08:47:06.990 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:06 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:06.753+0000 7f6d2722a640 -1 osd.6 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:47:07.690 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:07 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:07.350+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:47:07.690 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:07 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:07.350+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:30.627617+0000 front 2024-09-11T08:46:30.627656+0000 (oldest deadline 2024-09-11T08:46:56.527588+0000) 2024-09-11T08:47:07.690 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:07 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:07.350+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:40.529169+0000 front 2024-09-11T08:46:40.529077+0000 (oldest deadline 2024-09-11T08:47:01.628993+0000) 2024-09-11T08:47:07.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:07 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:07.689+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:07.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:07 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:07.689+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:07.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:07 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:07.689+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:08.714 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:08.393+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:47:08.714 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:08.393+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:30.627617+0000 front 2024-09-11T08:46:30.627656+0000 (oldest deadline 2024-09-11T08:46:56.527588+0000) 2024-09-11T08:47:08.714 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:08.393+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:40.529169+0000 front 2024-09-11T08:46:40.529077+0000 (oldest deadline 2024-09-11T08:47:01.628993+0000) 2024-09-11T08:47:08.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:08.714+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:08.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:08.714+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:08.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:08 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:08.714+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:09.740 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:09.429+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:47:09.741 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:09.429+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:30.627617+0000 front 2024-09-11T08:46:30.627656+0000 (oldest deadline 2024-09-11T08:46:56.527588+0000) 2024-09-11T08:47:09.741 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:09.429+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:40.529169+0000 front 2024-09-11T08:46:40.529077+0000 (oldest deadline 2024-09-11T08:47:01.628993+0000) 2024-09-11T08:47:10.240 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:09.751+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:10.241 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:09.751+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:10.241 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:09 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:09.751+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:10.729 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:10 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:10.430+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:47:10.729 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:10 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:10.430+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:30.627617+0000 front 2024-09-11T08:46:30.627656+0000 (oldest deadline 2024-09-11T08:46:56.527588+0000) 2024-09-11T08:47:10.729 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:10 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:10.430+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:40.529169+0000 front 2024-09-11T08:46:40.529077+0000 (oldest deadline 2024-09-11T08:47:01.628993+0000) 2024-09-11T08:47:10.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:10 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:10.728+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:10.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:10 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:10.728+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:10.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:10 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:10.728+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:11.736 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:11.471+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:21.926082+0000 front 2024-09-11T08:46:21.926139+0000 (oldest deadline 2024-09-11T08:46:46.626237+0000) 2024-09-11T08:47:11.737 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:11.471+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:30.627617+0000 front 2024-09-11T08:46:30.627656+0000 (oldest deadline 2024-09-11T08:46:56.527588+0000) 2024-09-11T08:47:11.737 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6[49184]: 2024-09-11T08:47:11.471+0000 7f6d22840640 -1 osd.6 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:40.529169+0000 front 2024-09-11T08:46:40.529077+0000 (oldest deadline 2024-09-11T08:47:01.628993+0000) 2024-09-11T08:47:11.890 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:47:11.628Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-11T08:47:11.891 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:11 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:47:11.629Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi189.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.189:8443: connect: connection refused" 2024-09-11T08:47:11.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:11.737+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:11.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:11.737+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:11.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:11 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:11.737+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:12.328 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:12 smithi189 podman[70467]: 2024-09-11 08:47:12.060203268 +0000 UTC m=+5.420026279 container died 978f79dec73da432ca9f04dad23ad7511bae410de12a987a8c2b1610f4289446 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T08:47:12.678 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:12 smithi189 podman[70467]: 2024-09-11 08:47:12.41135081 +0000 UTC m=+5.771173805 container cleanup 978f79dec73da432ca9f04dad23ad7511bae410de12a987a8c2b1610f4289446 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T08:47:12.678 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:12 smithi189 bash[70467]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6 2024-09-11T08:47:12.985 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:12 smithi189 podman[70482]: 2024-09-11 08:47:12.679285947 +0000 UTC m=+0.613231001 container remove 978f79dec73da432ca9f04dad23ad7511bae410de12a987a8c2b1610f4289446 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T08:47:12.986 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:12 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:12.784+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:12.986 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:12 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:12.784+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:12.986 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:12 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:12.784+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:12.986 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:12 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:12.784+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-11T08:46:47.795355+0000 front 2024-09-11T08:46:47.795358+0000 (oldest deadline 2024-09-11T08:47:11.895396+0000) 2024-09-11T08:47:13.240 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:12 smithi189 podman[70572]: 2024-09-11 08:47:12.889276483 +0000 UTC m=+0.016103637 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:47:13.240 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:13 smithi189 podman[70572]: 2024-09-11 08:47:13.019588332 +0000 UTC m=+0.146415520 container create f1379d7018f8348e064d1ba06e9c68b5e8a10f6b9a4f15dd203c4d746e55abd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T08:47:13.240 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:13 smithi189 podman[70572]: 2024-09-11 08:47:13.215960521 +0000 UTC m=+0.342787675 container init f1379d7018f8348e064d1ba06e9c68b5e8a10f6b9a4f15dd203c4d746e55abd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T08:47:13.240 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:13 smithi189 podman[70572]: 2024-09-11 08:47:13.219517954 +0000 UTC m=+0.346345104 container start f1379d7018f8348e064d1ba06e9c68b5e8a10f6b9a4f15dd203c4d746e55abd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-11T08:47:13.676 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:13 smithi189 podman[70572]: 2024-09-11 08:47:13.299417182 +0000 UTC m=+0.426244337 container attach f1379d7018f8348e064d1ba06e9c68b5e8a10f6b9a4f15dd203c4d746e55abd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T08:47:13.676 INFO:journalctl@ceph.osd.6.smithi189.stdout:Sep 11 08:47:13 smithi189 podman[70572]: 2024-09-11 08:47:13.392918521 +0000 UTC m=+0.519745668 container died f1379d7018f8348e064d1ba06e9c68b5e8a10f6b9a4f15dd203c4d746e55abd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T08:47:13.927 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:13 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:13.804+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:13.927 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:13 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:13.804+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:13.927 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:13 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:13.804+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:13.927 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:13 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:13.804+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-11T08:46:47.795355+0000 front 2024-09-11T08:46:47.795358+0000 (oldest deadline 2024-09-11T08:47:11.895396+0000) 2024-09-11T08:47:13.945 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T08:47:13.985 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:47:13.985 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T08:47:13.985 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T08:47:13.985 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.7 2024-09-11T08:47:14.240 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:14 smithi189 systemd[1]: Stopping Ceph osd.7 for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:47:14.740 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:14 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:14.318+0000 7fa9f3ec9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:47:14.740 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:14 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:14.318+0000 7fa9f3ec9640 -1 osd.7 66 *** Got signal Terminated *** 2024-09-11T08:47:14.740 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:14 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:14.318+0000 7fa9f3ec9640 -1 osd.7 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:47:15.240 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:14 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:14.773+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:15.240 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:14 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:14.773+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:15.240 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:14 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:14.773+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:15.240 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:14 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:14.773+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-11T08:46:47.795355+0000 front 2024-09-11T08:46:47.795358+0000 (oldest deadline 2024-09-11T08:47:11.895396+0000) 2024-09-11T08:47:15.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:15 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:15.726+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:15.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:15 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:15.726+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:15.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:15 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:15.726+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:15.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:15 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:15.726+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-11T08:46:47.795355+0000 front 2024-09-11T08:46:47.795358+0000 (oldest deadline 2024-09-11T08:47:11.895396+0000) 2024-09-11T08:47:17.240 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:16 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:16.772+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:17.240 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:16 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:16.772+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:17.240 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:16 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:16.772+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:17.240 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:16 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:16.772+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-11T08:46:47.795355+0000 front 2024-09-11T08:46:47.795358+0000 (oldest deadline 2024-09-11T08:47:11.895396+0000) 2024-09-11T08:47:17.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:17 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:17.731+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:17.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:17 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:17.731+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:17.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:17 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:17.731+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:17.990 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:17 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:17.731+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-11T08:46:47.795355+0000 front 2024-09-11T08:46:47.795358+0000 (oldest deadline 2024-09-11T08:47:11.895396+0000) 2024-09-11T08:47:19.217 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:18 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:18.775+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-11T08:46:23.192788+0000 front 2024-09-11T08:46:23.192780+0000 (oldest deadline 2024-09-11T08:46:47.892697+0000) 2024-09-11T08:47:19.218 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:18 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:18.775+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-11T08:46:31.393895+0000 front 2024-09-11T08:46:31.393866+0000 (oldest deadline 2024-09-11T08:46:56.093645+0000) 2024-09-11T08:47:19.218 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:18 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:18.775+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-11T08:46:36.094152+0000 front 2024-09-11T08:46:36.094080+0000 (oldest deadline 2024-09-11T08:47:01.994060+0000) 2024-09-11T08:47:19.218 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:18 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7[55043]: 2024-09-11T08:47:18.775+0000 7fa9f1ce4640 -1 osd.7 66 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-11T08:46:47.795355+0000 front 2024-09-11T08:46:47.795358+0000 (oldest deadline 2024-09-11T08:47:11.895396+0000) 2024-09-11T08:47:19.943 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:19 smithi189 podman[70675]: 2024-09-11 08:47:19.62191873 +0000 UTC m=+5.419187832 container died 401e3023ee2a0fc4e68231e1f31d676e1e9efc9e002a409776e7f533e52e1626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True) 2024-09-11T08:47:19.943 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:19 smithi189 podman[70675]: 2024-09-11 08:47:19.944079802 +0000 UTC m=+5.741348937 container cleanup 401e3023ee2a0fc4e68231e1f31d676e1e9efc9e002a409776e7f533e52e1626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-11T08:47:20.240 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:19 smithi189 bash[70675]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7 2024-09-11T08:47:20.240 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:20 smithi189 podman[70703]: 2024-09-11 08:47:20.120511305 +0000 UTC m=+0.498168215 container remove 401e3023ee2a0fc4e68231e1f31d676e1e9efc9e002a409776e7f533e52e1626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True) 2024-09-11T08:47:20.840 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:20 smithi189 podman[70796]: 2024-09-11 08:47:20.448469968 +0000 UTC m=+0.015783406 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:47:20.840 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:20 smithi189 podman[70796]: 2024-09-11 08:47:20.554711159 +0000 UTC m=+0.122024596 container create 4ae1e66b32bc31d63ca42a5dd2b1d713018d44688c6eea33af4fdf3b184645fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T08:47:20.840 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:20 smithi189 podman[70796]: 2024-09-11 08:47:20.757507363 +0000 UTC m=+0.324820802 container init 4ae1e66b32bc31d63ca42a5dd2b1d713018d44688c6eea33af4fdf3b184645fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T08:47:20.840 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:20 smithi189 podman[70796]: 2024-09-11 08:47:20.760775042 +0000 UTC m=+0.328088480 container start 4ae1e66b32bc31d63ca42a5dd2b1d713018d44688c6eea33af4fdf3b184645fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T08:47:20.840 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:20 smithi189 podman[70796]: 2024-09-11 08:47:20.840657179 +0000 UTC m=+0.407970632 container attach 4ae1e66b32bc31d63ca42a5dd2b1d713018d44688c6eea33af4fdf3b184645fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-11T08:47:21.226 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:20 smithi189 podman[70796]: 2024-09-11 08:47:20.939030612 +0000 UTC m=+0.506344051 container died 4ae1e66b32bc31d63ca42a5dd2b1d713018d44688c6eea33af4fdf3b184645fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:47:21.474 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T08:47:21.490 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:21 smithi189 podman[70819]: 2024-09-11 08:47:21.456724507 +0000 UTC m=+0.507854161 container remove 4ae1e66b32bc31d63ca42a5dd2b1d713018d44688c6eea33af4fdf3b184645fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-11T08:47:21.490 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:21 smithi189 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.7.service: Deactivated successfully. 2024-09-11T08:47:21.490 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:21 smithi189 systemd[1]: Stopped Ceph osd.7 for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:47:21.490 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 11 08:47:21 smithi189 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@osd.7.service: Consumed 4.800s CPU time. 2024-09-11T08:47:21.512 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:47:21.513 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T08:47:21.513 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-11T08:47:21.513 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-11T08:47:21.874 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 11 08:47:21 smithi131 systemd[1]: Stopping Ceph rgw.foo.a for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:47:21.874 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:21 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:47:21.628Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi189.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.189:8443: connect: connection refused" 2024-09-11T08:47:21.874 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:21 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:47:21.628Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-11T08:47:21.874 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:21 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:47:21.629Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi189.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.189:8443: connect: connection refused" 2024-09-11T08:47:22.140 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 11 08:47:21 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-rgw-foo-a[63800]: 2024-09-11T08:47:21.872+0000 7f03977fe640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:47:22.140 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 11 08:47:21 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-rgw-foo-a[63800]: 2024-09-11T08:47:21.873+0000 7f03a7dd77c0 -1 shutting down 2024-09-11T08:47:31.889 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 11 08:47:31 smithi131 bash[81247]: time="2024-09-11T08:47:31Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-11T08:47:31.893 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:31 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:47:31.628Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi189.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.189:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-11T08:47:31.893 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:31 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:47:31.629Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-11T08:47:31.893 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:31 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:47:31.630Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi189.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.189:8443: connect: connection refused" 2024-09-11T08:47:32.140 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 11 08:47:31 smithi131 podman[81247]: 2024-09-11 08:47:31.889380624 +0000 UTC m=+10.177629197 container died e5be8d02b62001f4a9274473cc7b13bbda90d7dfc1d06669274254a051b68246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-rgw-foo-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T08:47:32.524 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 11 08:47:32 smithi131 podman[81247]: 2024-09-11 08:47:32.315444706 +0000 UTC m=+10.603693278 container cleanup e5be8d02b62001f4a9274473cc7b13bbda90d7dfc1d06669274254a051b68246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:47:32.524 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 11 08:47:32 smithi131 bash[81247]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-11T08:47:32.737 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-11T08:47:32.772 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:47:32.773 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-11T08:47:32.773 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-11T08:47:32.773 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@prometheus.a 2024-09-11T08:47:33.067 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:47:32 smithi189 systemd[1]: Stopping Ceph prometheus.a for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:47:33.334 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:47:33 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:47:33.067Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:47:33.335 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:47:33 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:47:33.067Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T08:47:33.335 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:47:33 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:47:33.067Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T08:47:33.335 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:47:33 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:47:33.067Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T08:47:33.335 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:47:33 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:47:33.067Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T08:47:33.335 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:47:33 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:47:33.067Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T08:47:33.335 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:47:33 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:47:33.067Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T08:47:33.335 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:47:33 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:47:33.067Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T08:47:33.335 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:47:33 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:47:33.067Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T08:47:33.335 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:47:33 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:47:33.068Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T08:47:33.335 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:47:33 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:47:33.068Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T08:47:33.335 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:47:33 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a[64558]: ts=2024-09-11T08:47:33.068Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T08:47:33.335 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:47:33 smithi189 podman[70898]: 2024-09-11 08:47:33.077790495 +0000 UTC m=+0.112745540 container died ba70272d94b088e09ec5a083b722f34eb6076db4a28662de528e2cd4e60ed9a3 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:47:33.709 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:47:33 smithi189 podman[70898]: 2024-09-11 08:47:33.42544007 +0000 UTC m=+0.460395117 container cleanup ba70272d94b088e09ec5a083b722f34eb6076db4a28662de528e2cd4e60ed9a3 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:47:33.709 INFO:journalctl@ceph.prometheus.a.smithi189.stdout:Sep 11 08:47:33 smithi189 bash[70898]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T08:47:33.847 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-11T08:47:33.882 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:47:33.883 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-11T08:47:33.883 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm rm-cluster --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T08:47:35.766 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:35 smithi131 systemd[1]: Stopping Ceph alertmanager.a for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:47:35.766 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:35 smithi131 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a[72299]: ts=2024-09-11T08:47:35.746Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:47:36.029 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:35 smithi131 podman[81685]: 2024-09-11 08:47:35.766850184 +0000 UTC m=+0.117586623 container died f1618f18eebadd5067d5707a97da15c8f66bf589c9f3bb76e33369ebf50ced19 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:47:36.390 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:36 smithi131 podman[81685]: 2024-09-11 08:47:36.11284803 +0000 UTC m=+0.463584466 container cleanup f1618f18eebadd5067d5707a97da15c8f66bf589c9f3bb76e33369ebf50ced19 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:47:36.391 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:36 smithi131 bash[81685]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T08:47:36.671 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:36 smithi131 podman[81697]: 2024-09-11 08:47:36.3975069 +0000 UTC m=+0.625986226 container remove f1618f18eebadd5067d5707a97da15c8f66bf589c9f3bb76e33369ebf50ced19 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:47:36.671 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:36 smithi131 podman[81697]: 2024-09-11 08:47:36.481455078 +0000 UTC m=+0.709934405 volume remove da44ed3feab56fd10f01fd1d7fdab5a24367d0a1fde2655001c4033516cf25af 2024-09-11T08:47:37.119 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:47:36 smithi131 systemd[1]: Stopping Ceph node-exporter.a for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:47:37.119 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:36 smithi131 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T08:47:37.119 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:36 smithi131 systemd[1]: Stopped Ceph alertmanager.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:47:37.119 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 11 08:47:36 smithi131 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@alertmanager.a.service: Consumed 1.150s CPU time. 2024-09-11T08:47:37.384 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:47:37 smithi131 podman[81806]: 2024-09-11 08:47:37.12016359 +0000 UTC m=+0.119562815 container died 82e0910de1da02fb6403d51d8142224b44f4329c3f18ceb62edba2e63aa546cf (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:47:37.640 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:47:37 smithi131 podman[81806]: 2024-09-11 08:47:37.484369494 +0000 UTC m=+0.483768709 container cleanup 82e0910de1da02fb6403d51d8142224b44f4329c3f18ceb62edba2e63aa546cf (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:47:37.640 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:47:37 smithi131 bash[81806]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a 2024-09-11T08:47:38.041 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:47:37 smithi131 podman[81819]: 2024-09-11 08:47:37.74408031 +0000 UTC m=+0.627765298 container remove 82e0910de1da02fb6403d51d8142224b44f4329c3f18ceb62edba2e63aa546cf (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:47:38.041 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:47:37 smithi131 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:47:38.041 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:47:37 smithi131 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-11T08:47:38.041 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:47:37 smithi131 systemd[1]: Stopped Ceph node-exporter.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:47:38.041 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 11 08:47:37 smithi131 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 2.388s CPU time. 2024-09-11T08:47:38.515 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm rm-cluster --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T08:47:40.240 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:39 smithi189 systemd[1]: Stopping Ceph iscsi.iscsi.a for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:47:40.240 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:40 smithi189 ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a[59728]: debug Shutdown received 2024-09-11T08:47:50.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:50 smithi189 bash[71276]: time="2024-09-11T08:47:50Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-11T08:47:50.491 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:50 smithi189 podman[71276]: 2024-09-11 08:47:50.187032877 +0000 UTC m=+10.130417572 container died c06b1ffe67470c3e97d216cc2fec2d56e01d11c3b21ee5fc0fed0119f87a1652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-11T08:47:50.804 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:50 smithi189 podman[71276]: 2024-09-11 08:47:50.603858782 +0000 UTC m=+10.547243468 container cleanup c06b1ffe67470c3e97d216cc2fec2d56e01d11c3b21ee5fc0fed0119f87a1652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:47:50.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:50 smithi189 bash[71276]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-11T08:47:51.126 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:50 smithi189 podman[71290]: 2024-09-11 08:47:50.805583252 +0000 UTC m=+0.613469215 container remove c06b1ffe67470c3e97d216cc2fec2d56e01d11c3b21ee5fc0fed0119f87a1652 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-11T08:47:51.126 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:50 smithi189 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-11T08:47:51.482 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:51 smithi189 podman[71334]: 2024-09-11 08:47:51.127105487 +0000 UTC m=+0.110194954 container died c57800cb1cda8ef7be4b7dc897b899bfa5d7cc9bb640ff540d7b489a5176acac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=) 2024-09-11T08:47:51.483 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:51 smithi189 podman[71334]: 2024-09-11 08:47:51.483728931 +0000 UTC m=+0.466818389 container cleanup c57800cb1cda8ef7be4b7dc897b899bfa5d7cc9bb640ff540d7b489a5176acac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-09-11T08:47:51.734 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:51 smithi189 bash[71334]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-11T08:47:51.734 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:51 smithi189 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-11T08:47:51.734 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:51 smithi189 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 59659 (conmon) remains running after unit stopped. 2024-09-11T08:47:51.734 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:51 smithi189 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 71346 (podman) remains running after unit stopped. 2024-09-11T08:47:51.735 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:51 smithi189 systemd[1]: Stopped Ceph iscsi.iscsi.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:47:51.735 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:51 smithi189 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 2.588s CPU time. 2024-09-11T08:47:51.990 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:47:51 smithi189 systemd[1]: Stopping Ceph grafana.a for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:47:51.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi189.stdout:Sep 11 08:47:51 smithi189 podman[71346]: 2024-09-11 08:47:51.735202955 +0000 UTC m=+0.604653613 container remove c57800cb1cda8ef7be4b7dc897b899bfa5d7cc9bb640ff540d7b489a5176acac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:47:52.396 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:47:52 smithi189 podman[71467]: 2024-09-11 08:47:52.140916407 +0000 UTC m=+0.103893010 container died 46bbf925bf21cee5be47aaff9c563e4adecfee1a0124c055fd3595c02d14b4c6 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a, io.openshift.expose-services=, maintainer=Paul Cuzner , io.buildah.version=1.32.0, version=8.5, distribution-scope=public, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.display-name=Red Hat Universal Base Image 8, description=Ceph Grafana Container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.tags=base rhel8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897, com.redhat.component=ubi8-container, release=239.1651231664, architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git) 2024-09-11T08:47:52.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:47:52 smithi189 podman[71467]: 2024-09-11 08:47:52.479662045 +0000 UTC m=+0.442638645 container cleanup 46bbf925bf21cee5be47aaff9c563e4adecfee1a0124c055fd3595c02d14b4c6 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a, version=8.5, distribution-scope=public, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, name=ubi8, architecture=x86_64, io.openshift.expose-services=, build-date=2022-04-29T11:48:11.055897, io.openshift.tags=base rhel8, release=239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , vendor=Red Hat, Inc., com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, description=Ceph Grafana Container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, io.buildah.version=1.32.0) 2024-09-11T08:47:52.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:47:52 smithi189 bash[71467]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a 2024-09-11T08:47:52.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:47:52 smithi189 podman[71481]: 2024-09-11 08:47:52.647505984 +0000 UTC m=+0.509269735 container remove 46bbf925bf21cee5be47aaff9c563e4adecfee1a0124c055fd3595c02d14b4c6 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-grafana-a, io.openshift.expose-services=, name=ubi8, release=239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.tags=base rhel8, com.redhat.component=ubi8-container, vendor=Red Hat, Inc., version=8.5, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner , io.buildah.version=1.32.0, io.k8s.display-name=Red Hat Universal Base Image 8, distribution-scope=public, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897) 2024-09-11T08:47:52.649 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:47:52 smithi189 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:47:53.240 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:47:53 smithi189 systemd[1]: Stopping Ceph node-exporter.b for 50e4d0bc-7018-11ef-bcea-c7b262605968... 2024-09-11T08:47:53.240 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:47:52 smithi189 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-11T08:47:53.240 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:47:52 smithi189 systemd[1]: Stopped Ceph grafana.a for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:47:53.240 INFO:journalctl@ceph.grafana.a.smithi189.stdout:Sep 11 08:47:52 smithi189 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@grafana.a.service: Consumed 6.772s CPU time. 2024-09-11T08:47:53.640 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:47:53 smithi189 podman[71597]: 2024-09-11 08:47:53.361025475 +0000 UTC m=+0.100065081 container died 4df34c86104491bfcf8f08866d3dfeefa652889703edc6a35fd308a027ea697c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:47:53.640 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:47:53 smithi189 podman[71597]: 2024-09-11 08:47:53.64163681 +0000 UTC m=+0.380676416 container cleanup 4df34c86104491bfcf8f08866d3dfeefa652889703edc6a35fd308a027ea697c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:47:53.984 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:47:53 smithi189 bash[71597]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b 2024-09-11T08:47:53.985 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:47:53 smithi189 podman[71609]: 2024-09-11 08:47:53.876425412 +0000 UTC m=+0.517072687 container remove 4df34c86104491bfcf8f08866d3dfeefa652889703edc6a35fd308a027ea697c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-50e4d0bc-7018-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:47:53.985 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:47:53 smithi189 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:47:54.326 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:47:54 smithi189 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-11T08:47:54.326 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:47:54 smithi189 systemd[1]: Stopped Ceph node-exporter.b for 50e4d0bc-7018-11ef-bcea-c7b262605968. 2024-09-11T08:47:54.326 INFO:journalctl@ceph.node-exporter.b.smithi189.stdout:Sep 11 08:47:54 smithi189 systemd[1]: ceph-50e4d0bc-7018-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 2.414s CPU time. 2024-09-11T08:47:54.642 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:47:54.675 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:47:54.705 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T08:47:54.707 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901292/remote/smithi131/crash 2024-09-11T08:47:54.707 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/crash -- . 2024-09-11T08:47:54.786 INFO:teuthology.orchestra.run.smithi131.stderr:tar: /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T08:47:54.786 INFO:teuthology.orchestra.run.smithi131.stderr:tar: Error is not recoverable: exiting now 2024-09-11T08:47:54.788 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901292/remote/smithi189/crash 2024-09-11T08:47:54.789 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/crash -- . 2024-09-11T08:47:54.821 INFO:teuthology.orchestra.run.smithi189.stderr:tar: /var/lib/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T08:47:54.821 INFO:teuthology.orchestra.run.smithi189.stderr:tar: Error is not recoverable: exiting now 2024-09-11T08:47:54.822 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T08:47:54.822 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-11T08:47:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:grep: /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-11T08:47:54.855 INFO:tasks.cephadm:Compressing logs... 2024-09-11T08:47:54.855 DEBUG:teuthology.orchestra.run.smithi131:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T08:47:54.897 DEBUG:teuthology.orchestra.run.smithi189:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T08:47:54.923 INFO:teuthology.orchestra.run.smithi189.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-11T08:47:54.923 INFO:teuthology.orchestra.run.smithi189.stderr:: No such file or directory 2024-09-11T08:47:54.923 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T08:47:54.923 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/tcmu-runner.log: gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T08:47:54.923 INFO:teuthology.orchestra.run.smithi189.stderr: 63.5% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-11T08:47:54.923 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T08:47:54.923 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T08:47:54.926 INFO:teuthology.orchestra.run.smithi131.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T08:47:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:: No such file or directory 2024-09-11T08:47:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T08:47:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T08:47:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T08:47:54.927 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T08:47:54.927 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T08:47:54.928 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T08:47:54.928 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T08:47:54.928 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T08:47:54.928 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T08:47:54.930 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T08:47:54.930 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T08:47:54.934 INFO:teuthology.orchestra.run.smithi189.stderr: 91.2% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T08:47:54.936 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.2.log: 91.1% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T08:47:54.936 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-11T08:47:54.937 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T08:47:54.937 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: 58.7% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-11T08:47:54.938 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T08:47:54.949 INFO:teuthology.orchestra.run.smithi189.stderr: 88.4% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T08:47:54.960 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 91.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T08:47:55.144 INFO:teuthology.orchestra.run.smithi189.stderr: 91.5% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T08:47:55.144 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T08:47:55.161 INFO:teuthology.orchestra.run.smithi131.stderr: 91.7% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T08:47:55.215 INFO:teuthology.orchestra.run.smithi189.stderr: 92.8% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T08:47:55.272 INFO:teuthology.orchestra.run.smithi189.stderr: 93.1% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T08:47:55.341 INFO:teuthology.orchestra.run.smithi189.stderr: 92.9% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T08:47:55.355 INFO:teuthology.orchestra.run.smithi131.stderr: 92.7% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T08:47:55.367 INFO:teuthology.orchestra.run.smithi189.stderr: 93.0% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T08:47:55.369 INFO:teuthology.orchestra.run.smithi189.stderr: 2024-09-11T08:47:55.369 INFO:teuthology.orchestra.run.smithi189.stderr:real 0m0.457s 2024-09-11T08:47:55.369 INFO:teuthology.orchestra.run.smithi189.stderr:user 0m1.696s 2024-09-11T08:47:55.369 INFO:teuthology.orchestra.run.smithi189.stderr:sys 0m0.092s 2024-09-11T08:47:55.399 INFO:teuthology.orchestra.run.smithi131.stderr: 93.1% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T08:47:55.413 INFO:teuthology.orchestra.run.smithi131.stderr: 92.9% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T08:47:55.419 INFO:teuthology.orchestra.run.smithi131.stderr: 92.8% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T08:47:55.568 INFO:teuthology.orchestra.run.smithi131.stderr: 90.8% -- replaced with /var/log/ceph/50e4d0bc-7018-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T08:47:55.570 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-11T08:47:55.570 INFO:teuthology.orchestra.run.smithi131.stderr:real 0m0.659s 2024-09-11T08:47:55.570 INFO:teuthology.orchestra.run.smithi131.stderr:user 0m2.865s 2024-09-11T08:47:55.570 INFO:teuthology.orchestra.run.smithi131.stderr:sys 0m0.138s 2024-09-11T08:47:55.570 INFO:tasks.cephadm:Archiving logs... 2024-09-11T08:47:55.571 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901292/remote/smithi131/log 2024-09-11T08:47:55.572 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T08:47:56.027 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901292/remote/smithi189/log 2024-09-11T08:47:56.028 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T08:47:56.316 INFO:tasks.cephadm:Removing cluster... 2024-09-11T08:47:56.316 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm rm-cluster --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 --force 2024-09-11T08:47:56.769 DEBUG:teuthology.orchestra.run.smithi189:> sudo cephadm rm-cluster --fsid 50e4d0bc-7018-11ef-bcea-c7b262605968 --force 2024-09-11T08:47:57.252 INFO:tasks.cephadm:Teardown complete 2024-09-11T08:47:57.252 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T08:47:57.260 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T08:47:57.261 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T08:47:57.263 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T08:47:57.338 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T08:47:57.346 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T08:47:57.346 DEBUG:teuthology.orchestra.run.smithi131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:47:57.349 DEBUG:teuthology.orchestra.run.smithi189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:47:57.361 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:47:57.363 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:47:57.367 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:47:57.367 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-09-11T08:47:57.367 INFO:teuthology.orchestra.run.smithi131.stdout:^* hv01.front.sepia.ceph.com 3 6 377 55 -2026us[-2044us] +/- 56ms 2024-09-11T08:47:57.367 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 121 +1740us[+1737us] +/- 77ms 2024-09-11T08:47:57.367 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +952us[ +935us] +/- 65ms 2024-09-11T08:47:57.367 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:47:57.369 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:47:57.369 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-09-11T08:47:57.369 INFO:teuthology.orchestra.run.smithi189.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 -2063us[-2028us] +/- 57ms 2024-09-11T08:47:57.369 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 120 +1691us[+1745us] +/- 77ms 2024-09-11T08:47:57.369 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 +1025us[+1060us] +/- 65ms 2024-09-11T08:47:57.369 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:47:57.369 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T08:47:57.377 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T08:47:57.378 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T08:47:57.387 DEBUG:teuthology.orchestra.run.smithi131:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T08:47:57.482 DEBUG:teuthology.orchestra.run.smithi189:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T08:47:57.553 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:47:57.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:57.582 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:47:57.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:57.610 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T08:47:57.619 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T08:47:57.628 INFO:teuthology.task.internal:Duration was 1327.559830 seconds 2024-09-11T08:47:57.628 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T08:47:57.636 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T08:47:57.637 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T08:47:57.639 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T08:47:57.675 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:47:57.690 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:47:58.030 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T08:47:58.030 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi131.front.sepia.ceph.com 2024-09-11T08:47:58.030 DEBUG:teuthology.orchestra.run.smithi131:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T08:47:58.051 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi189.front.sepia.ceph.com 2024-09-11T08:47:58.051 DEBUG:teuthology.orchestra.run.smithi189:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T08:47:58.095 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T08:47:58.095 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T08:47:58.098 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T08:47:58.162 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T08:47:58.163 DEBUG:teuthology.orchestra.run.smithi131:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T08:47:58.165 DEBUG:teuthology.orchestra.run.smithi189:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T08:47:58.436 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T08:47:58.446 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T08:47:58.446 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T08:47:58.471 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T08:47:58.497 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T08:47:58.507 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:47:58.514 DEBUG:teuthology.orchestra.run.smithi189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:47:58.535 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2024-09-11T08:47:58.561 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = core 2024-09-11T08:47:58.576 DEBUG:teuthology.orchestra.run.smithi131:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:47:58.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:58.626 DEBUG:teuthology.orchestra.run.smithi189:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:47:58.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:47:58.642 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T08:47:58.654 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T08:47:58.655 DEBUG:teuthology.misc:Transferring archived files from smithi131:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901292/remote/smithi131 2024-09-11T08:47:58.655 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T08:47:58.707 DEBUG:teuthology.misc:Transferring archived files from smithi189:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901292/remote/smithi189 2024-09-11T08:47:58.708 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T08:47:58.746 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T08:47:58.746 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T08:47:58.749 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T08:47:58.795 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T08:47:58.806 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T08:47:58.806 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T08:47:58.837 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T08:47:58.838 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T08:47:58.840 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T08:47:58.852 INFO:teuthology.orchestra.run.smithi131.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 08:47 /home/ubuntu/cephtest 2024-09-11T08:47:58.855 INFO:teuthology.orchestra.run.smithi189.stdout: 265340 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 08:47 /home/ubuntu/cephtest 2024-09-11T08:47:58.857 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T08:47:58.867 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T08:47:58.888 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} duration: 1327.5598299503326 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-11T08:47:58.889 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T08:47:58.982 INFO:teuthology.run:pass