2024-09-11T07:14:50.207 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T07:14:50.221 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T07:14:50.303 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901214 branch: wip-guits-quincy-2024-09-10-2008-quincy description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} email: gabrioux@ibm.com first_in_suite: false job_id: '7901214' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-10-2008-quincy ceph: conf: global: mon election default strategy: 1 ms bind msgr1: false ms bind msgr2: true ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 extra_packages: - cephadm selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-quincy-2024-09-10-2008-quincy sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 2282 sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-quincy-2024-09-10-2008-quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 targets: smithi143.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMd8DP0Hzxwcbj/sI9nmyffR/u31Fp+g5sTPl+z9BFsD+NxVTAzhnpkH6LEJhkKJ9ZzJWJn2fnLdawMokmSjqe8= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMDTTrX5mriZdxQKbi2jDp9zEXht91wuro/J7dT1vBwx5cDMLu8Lpod44+PDM5E0M6+0bwXXk/MH+yGzN3G+9cs= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - "set -ex\nfor f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x\n\ do\n echo \"rotating key for $f\"\n K=$(ceph auth get-key $f)\n NK=\"\ $K\"\n ceph orch daemon rotate-key $f\n while [ \"$K\" == \"$NK\" ]; do\n\ \ sleep 5\n NK=$(ceph auth get-key $f)\n done\ndone\n" teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_06:53:43 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T07:14:50.303 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa; will attempt to use it 2024-09-11T07:14:50.304 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa/tasks 2024-09-11T07:14:50.304 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T07:14:50.305 INFO:teuthology.task.internal:Checking packages... 2024-09-11T07:14:50.326 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd0e6828a2016d48cf25ad84064e50742bb1c39b9' 2024-09-11T07:14:50.326 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T07:14:50.326 INFO:teuthology.packaging:ref: None 2024-09-11T07:14:50.326 INFO:teuthology.packaging:tag: None 2024-09-11T07:14:50.326 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T07:14:50.326 INFO:teuthology.packaging:sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:14:50.326 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T07:14:50.472 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1623.gd0e6828a 2024-09-11T07:14:50.474 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T07:14:50.483 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T07:14:50.483 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T07:14:50.522 INFO:teuthology.task.internal:Saving configuration 2024-09-11T07:14:50.533 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T07:14:50.598 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T07:14:50.623 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901214', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 07:09:44.443576', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMd8DP0Hzxwcbj/sI9nmyffR/u31Fp+g5sTPl+z9BFsD+NxVTAzhnpkH6LEJhkKJ9ZzJWJn2fnLdawMokmSjqe8='} 2024-09-11T07:14:50.644 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901214', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 07:09:44.445343', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMDTTrX5mriZdxQKbi2jDp9zEXht91wuro/J7dT1vBwx5cDMLu8Lpod44+PDM5E0M6+0bwXXk/MH+yGzN3G+9cs='} 2024-09-11T07:14:50.644 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T07:14:50.653 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-11T07:14:50.653 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-11T07:14:50.654 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T07:14:50.742 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f71618b3ac0>, signals=[15]) 2024-09-11T07:14:50.743 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T07:14:50.755 INFO:teuthology.task.internal:Opening connections... 2024-09-11T07:14:50.755 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi143.front.sepia.ceph.com 2024-09-11T07:14:50.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:14:50.841 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-09-11T07:14:50.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:14:50.922 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T07:14:50.931 DEBUG:teuthology.orchestra.run.smithi143:> uname -m 2024-09-11T07:14:50.949 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2024-09-11T07:14:50.949 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release 2024-09-11T07:14:51.007 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="CentOS Stream" 2024-09-11T07:14:51.007 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="9" 2024-09-11T07:14:51.007 INFO:teuthology.orchestra.run.smithi143.stdout:ID="centos" 2024-09-11T07:14:51.007 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="rhel fedora" 2024-09-11T07:14:51.007 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="9" 2024-09-11T07:14:51.008 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el9" 2024-09-11T07:14:51.008 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T07:14:51.008 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31" 2024-09-11T07:14:51.008 INFO:teuthology.orchestra.run.smithi143.stdout:LOGO="fedora-logo-icon" 2024-09-11T07:14:51.008 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T07:14:51.008 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://centos.org/" 2024-09-11T07:14:51.008 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T07:14:51.008 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T07:14:51.008 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T07:14:51.009 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2024-09-11T07:14:51.032 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-09-11T07:14:51.049 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-09-11T07:14:51.050 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-09-11T07:14:51.106 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-09-11T07:14:51.106 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-09-11T07:14:51.106 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-09-11T07:14:51.107 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-09-11T07:14:51.107 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-09-11T07:14:51.107 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-09-11T07:14:51.107 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T07:14:51.107 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-09-11T07:14:51.107 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-09-11T07:14:51.107 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T07:14:51.107 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-09-11T07:14:51.107 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T07:14:51.107 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T07:14:51.107 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T07:14:51.107 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-09-11T07:14:51.123 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T07:14:51.135 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T07:14:51.155 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T07:14:51.155 DEBUG:teuthology.orchestra.run.smithi143:> test '!' -e /home/ubuntu/cephtest 2024-09-11T07:14:51.158 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-09-11T07:14:51.173 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T07:14:51.188 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T07:14:51.189 DEBUG:teuthology.orchestra.run.smithi143:> test -z $(ls -A /var/lib/ceph) 2024-09-11T07:14:51.215 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-09-11T07:14:51.250 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T07:14:51.306 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T07:14:51.325 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T07:14:51.325 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-11T07:14:51.325 DEBUG:teuthology.orchestra.run.smithi143:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T07:14:51.326 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T07:14:51.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:14:51.342 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-09-11T07:14:51.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:14:51.344 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2024-09-11T07:14:51.399 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-503.el9.x86_64 2024-09-11T07:14:51.399 INFO:teuthology.task.kernel:Running kernel on smithi164: 5.14.0-503.el9.x86_64 2024-09-11T07:14:51.399 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum install -y kernel 2024-09-11T07:14:51.401 INFO:teuthology.orchestra.run.smithi143.stdout:5.14.0-503.el9.x86_64 2024-09-11T07:14:51.401 INFO:teuthology.task.kernel:Running kernel on smithi143: 5.14.0-503.el9.x86_64 2024-09-11T07:14:51.401 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum install -y kernel 2024-09-11T07:14:53.715 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-11T07:14:53.970 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-11T07:14:56.941 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 9.8 MB/s | 20 MB 00:02 2024-09-11T07:14:58.754 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - AppStream 5.5 MB/s | 20 MB 00:03 2024-09-11T07:15:01.833 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 6.5 MB 00:00 2024-09-11T07:15:03.048 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-11T07:15:03.317 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 19 kB 00:00 2024-09-11T07:15:04.529 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-11T07:15:04.664 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-11T07:15:05.785 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-11T07:15:10.010 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-09-11T07:15:11.408 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-11T07:15:11.983 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T07:15:11.983 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T07:15:12.077 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:15:12.085 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:15:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-09-11T07:15:12.086 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:15:12.086 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:15:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T07:15:12.086 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-11T07:15:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T07:15:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T07:15:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T07:15:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:15:12.086 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:15:12.086 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:15:12.086 INFO:teuthology.orchestra.run.smithi164.stdout:Install 4 Packages 2024-09-11T07:15:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:15:12.087 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 79 M 2024-09-11T07:15:12.087 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 126 M 2024-09-11T07:15:12.087 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:15:12.278 INFO:teuthology.orchestra.run.smithi164.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 264 kB/s | 23 kB 00:00 2024-09-11T07:15:13.381 INFO:teuthology.orchestra.run.smithi143.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T07:15:13.381 INFO:teuthology.orchestra.run.smithi143.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T07:15:13.465 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:15:13.477 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:15:13.477 INFO:teuthology.orchestra.run.smithi143.stdout: Package Architecture Version Repository Size 2024-09-11T07:15:13.477 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:15:13.477 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:15:13.477 INFO:teuthology.orchestra.run.smithi143.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T07:15:13.477 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-09-11T07:15:13.477 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T07:15:13.477 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T07:15:13.477 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T07:15:13.477 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:15:13.478 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:15:13.478 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:15:13.478 INFO:teuthology.orchestra.run.smithi143.stdout:Install 4 Packages 2024-09-11T07:15:13.478 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:15:13.478 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 79 M 2024-09-11T07:15:13.478 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 126 M 2024-09-11T07:15:13.478 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:15:13.793 INFO:teuthology.orchestra.run.smithi143.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 346 kB/s | 23 kB 00:00 2024-09-11T07:15:14.986 INFO:teuthology.orchestra.run.smithi164.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 5.6 MB/s | 16 MB 00:02 2024-09-11T07:15:17.036 INFO:teuthology.orchestra.run.smithi164.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 6.0 MB/s | 29 MB 00:04 2024-09-11T07:15:17.826 INFO:teuthology.orchestra.run.smithi143.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 3.8 MB/s | 16 MB 00:04 2024-09-11T07:15:20.003 INFO:teuthology.orchestra.run.smithi164.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 4.4 MB/s | 35 MB 00:07 2024-09-11T07:15:20.003 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:15:20.003 INFO:teuthology.orchestra.run.smithi164.stdout:Total 9.9 MB/s | 79 MB 00:07 2024-09-11T07:15:20.686 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:15:20.943 INFO:teuthology.orchestra.run.smithi143.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 4.8 MB/s | 35 MB 00:07 2024-09-11T07:15:21.044 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:15:21.044 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:15:21.226 INFO:teuthology.orchestra.run.smithi143.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 3.8 MB/s | 29 MB 00:07 2024-09-11T07:15:21.226 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:15:21.227 INFO:teuthology.orchestra.run.smithi143.stdout:Total 10 MB/s | 79 MB 00:07 2024-09-11T07:15:21.907 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:15:21.990 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:15:21.990 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:15:22.274 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:15:22.274 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:15:23.256 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:15:23.256 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:15:24.461 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:15:25.102 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:15:25.203 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:15:25.696 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:15:26.425 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:15:26.506 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:15:28.009 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:15:28.173 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:15:29.305 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:15:29.439 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:15:32.988 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:15:33.022 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:15:34.254 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:15:34.288 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:15:37.863 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:15:39.127 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:16:16.039 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:16:16.078 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:16:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:16:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:16:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:16:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:16:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:16:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:16:17.315 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:16:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T07:16:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T07:16:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T07:16:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T07:16:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:16:17.315 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:16:17.667 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:16:17.764 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:16:17.764 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:16:17.764 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T07:16:17.764 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T07:16:17.765 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T07:16:17.765 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-11T07:16:17.765 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi164.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T07:16:17.765 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:16:17.824 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:16:17.824 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:16:17.824 DEBUG:teuthology.orchestra.run.smithi164:> sudo rpm -qi grub2-tools 2024-09-11T07:16:17.855 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:16:17.856 INFO:teuthology.orchestra.run.smithi164.stdout:Name : grub2-tools 2024-09-11T07:16:17.856 INFO:teuthology.orchestra.run.smithi164.stdout:Epoch : 1 2024-09-11T07:16:17.856 INFO:teuthology.orchestra.run.smithi164.stdout:Version : 2.06 2024-09-11T07:16:17.856 INFO:teuthology.orchestra.run.smithi164.stdout:Release : 82.el9 2024-09-11T07:16:17.856 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: x86_64 2024-09-11T07:16:17.857 INFO:teuthology.orchestra.run.smithi164.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T07:16:17.857 INFO:teuthology.orchestra.run.smithi164.stdout:Group : Unspecified 2024-09-11T07:16:17.857 INFO:teuthology.orchestra.run.smithi164.stdout:Size : 8274670 2024-09-11T07:16:17.857 INFO:teuthology.orchestra.run.smithi164.stdout:License : GPLv3+ 2024-09-11T07:16:17.857 INFO:teuthology.orchestra.run.smithi164.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T07:16:17.857 INFO:teuthology.orchestra.run.smithi164.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T07:16:17.857 INFO:teuthology.orchestra.run.smithi164.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T07:16:17.857 INFO:teuthology.orchestra.run.smithi164.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T07:16:17.857 INFO:teuthology.orchestra.run.smithi164.stdout:Packager : builder@centos.org 2024-09-11T07:16:17.857 INFO:teuthology.orchestra.run.smithi164.stdout:Vendor : CentOS 2024-09-11T07:16:17.857 INFO:teuthology.orchestra.run.smithi164.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T07:16:17.857 INFO:teuthology.orchestra.run.smithi164.stdout:Summary : Support tools for GRUB. 2024-09-11T07:16:17.857 INFO:teuthology.orchestra.run.smithi164.stdout:Description : 2024-09-11T07:16:17.857 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:16:17.858 INFO:teuthology.orchestra.run.smithi164.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T07:16:17.858 INFO:teuthology.orchestra.run.smithi164.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T07:16:17.858 INFO:teuthology.orchestra.run.smithi164.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T07:16:17.858 INFO:teuthology.orchestra.run.smithi164.stdout:hardware devices. 2024-09-11T07:16:17.858 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:16:17.858 INFO:teuthology.orchestra.run.smithi164.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T07:16:17.859 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T07:16:17.859 INFO:teuthology.task.kernel:Updating grub on smithi164 to boot 5.14.0-505.el9.x86_64 2024-09-11T07:16:17.859 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T07:16:17.894 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:16:18.261 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:16:18.261 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:16:18.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:16:18.262 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:16:18.554 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2024-09-11T07:16:19.095 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:16:19.096 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:16:19.096 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:16:19.096 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T07:16:19.096 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T07:16:19.096 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T07:16:19.096 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T07:16:19.096 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:16:19.096 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:16:19.434 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:16:19.519 INFO:teuthology.orchestra.run.smithi143.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:16:19.520 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:16:19.520 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T07:16:19.520 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T07:16:19.520 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T07:16:19.520 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T07:16:19.520 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi143.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T07:16:19.520 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:16:19.579 INFO:teuthology.orchestra.run.smithi143.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:16:19.579 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:16:19.580 DEBUG:teuthology.orchestra.run.smithi143:> sudo rpm -qi grub2-tools 2024-09-11T07:16:19.621 INFO:teuthology.orchestra.run.smithi143.stdout:Name : grub2-tools 2024-09-11T07:16:19.621 INFO:teuthology.orchestra.run.smithi143.stdout:Epoch : 1 2024-09-11T07:16:19.621 INFO:teuthology.orchestra.run.smithi143.stdout:Version : 2.06 2024-09-11T07:16:19.621 INFO:teuthology.orchestra.run.smithi143.stdout:Release : 82.el9 2024-09-11T07:16:19.621 INFO:teuthology.orchestra.run.smithi143.stdout:Architecture: x86_64 2024-09-11T07:16:19.621 INFO:teuthology.orchestra.run.smithi143.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T07:16:19.621 INFO:teuthology.orchestra.run.smithi143.stdout:Group : Unspecified 2024-09-11T07:16:19.621 INFO:teuthology.orchestra.run.smithi143.stdout:Size : 8274670 2024-09-11T07:16:19.621 INFO:teuthology.orchestra.run.smithi143.stdout:License : GPLv3+ 2024-09-11T07:16:19.621 INFO:teuthology.orchestra.run.smithi143.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T07:16:19.621 INFO:teuthology.orchestra.run.smithi143.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T07:16:19.622 INFO:teuthology.orchestra.run.smithi143.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T07:16:19.622 INFO:teuthology.orchestra.run.smithi143.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T07:16:19.622 INFO:teuthology.orchestra.run.smithi143.stdout:Packager : builder@centos.org 2024-09-11T07:16:19.622 INFO:teuthology.orchestra.run.smithi143.stdout:Vendor : CentOS 2024-09-11T07:16:19.622 INFO:teuthology.orchestra.run.smithi143.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T07:16:19.622 INFO:teuthology.orchestra.run.smithi143.stdout:Summary : Support tools for GRUB. 2024-09-11T07:16:19.622 INFO:teuthology.orchestra.run.smithi143.stdout:Description : 2024-09-11T07:16:19.622 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:16:19.622 INFO:teuthology.orchestra.run.smithi143.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T07:16:19.622 INFO:teuthology.orchestra.run.smithi143.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T07:16:19.622 INFO:teuthology.orchestra.run.smithi143.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T07:16:19.622 INFO:teuthology.orchestra.run.smithi143.stdout:hardware devices. 2024-09-11T07:16:19.622 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:16:19.622 INFO:teuthology.orchestra.run.smithi143.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T07:16:19.623 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T07:16:19.623 INFO:teuthology.task.kernel:Updating grub on smithi143 to boot 5.14.0-505.el9.x86_64 2024-09-11T07:16:19.623 DEBUG:teuthology.orchestra.run.smithi143:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T07:16:19.846 INFO:teuthology.orchestra.run.smithi164.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T07:16:19.880 INFO:teuthology.orchestra.run.smithi164.stderr:done 2024-09-11T07:16:19.882 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-09-11T07:16:19.897 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.ZUOx3sH5WF 2024-09-11T07:16:19.898 DEBUG:teuthology.orchestra.run.smithi164:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ZUOx3sH5WF 2024-09-11T07:16:19.964 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.ZUOx3sH5WF 2024-09-11T07:16:20.104 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.ZUOx3sH5WF is 6KB 2024-09-11T07:16:20.154 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.ZUOx3sH5WF 2024-09-11T07:16:20.169 DEBUG:teuthology.orchestra.run.smithi164:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T07:16:20.239 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T07:16:20.240 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T07:16:20.240 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T07:16:20.240 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T07:16:20.241 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T07:16:20.274 INFO:teuthology.orchestra.run.smithi143.stderr:Generating grub configuration file ... 2024-09-11T07:16:20.406 DEBUG:teuthology.orchestra.run.smithi164:> sudo shutdown -r now 2024-09-11T07:16:21.648 INFO:teuthology.orchestra.run.smithi143.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T07:16:21.676 INFO:teuthology.orchestra.run.smithi143.stderr:done 2024-09-11T07:16:21.678 DEBUG:teuthology.orchestra.run.smithi143:> mktemp 2024-09-11T07:16:21.694 INFO:teuthology.orchestra.run.smithi143.stdout:/tmp/tmp.nWxNQILRMX 2024-09-11T07:16:21.695 DEBUG:teuthology.orchestra.run.smithi143:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nWxNQILRMX 2024-09-11T07:16:21.762 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0666 /tmp/tmp.nWxNQILRMX 2024-09-11T07:16:21.898 DEBUG:teuthology.orchestra.remote:smithi143:/tmp/tmp.nWxNQILRMX is 6KB 2024-09-11T07:16:21.946 DEBUG:teuthology.orchestra.run.smithi143:> rm -fr /tmp/tmp.nWxNQILRMX 2024-09-11T07:16:21.961 DEBUG:teuthology.orchestra.run.smithi143:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T07:16:22.025 INFO:teuthology.orchestra.run.smithi143.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T07:16:22.025 INFO:teuthology.orchestra.run.smithi143.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T07:16:22.025 INFO:teuthology.orchestra.run.smithi143.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T07:16:22.025 INFO:teuthology.orchestra.run.smithi143.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T07:16:22.026 DEBUG:teuthology.orchestra.run.smithi143:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T07:16:22.178 DEBUG:teuthology.orchestra.run.smithi143:> sudo shutdown -r now 2024-09-11T07:16:50.437 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-11T07:16:50.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-11T07:16:50.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:16:52.182 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T07:16:52.182 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-09-11T07:16:52.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:17:24.926 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-09-11T07:17:26.717 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-09-11T07:17:33.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-11T07:17:33.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:17:35.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-09-11T07:17:35.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:17:52.317 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-09-11T07:17:54.109 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-09-11T07:18:04.325 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-11T07:18:04.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:18:06.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-09-11T07:18:06.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:18:06.459 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-09-11T07:18:06.865 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-09-11T07:18:06.865 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-11T07:18:06.865 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2024-09-11T07:18:06.880 INFO:teuthology.orchestra.run.smithi143.stdout:5.14.0-505.el9.x86_64 2024-09-11T07:18:06.880 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T07:18:06.880 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T07:18:06.880 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T07:18:06.977 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-11T07:18:07.413 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-11T07:18:07.413 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-11T07:18:07.413 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-09-11T07:18:07.431 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-505.el9.x86_64 2024-09-11T07:18:07.431 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T07:18:07.431 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T07:18:07.431 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T07:18:07.881 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T07:18:07.881 DEBUG:teuthology.orchestra.run.smithi143:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T07:18:07.961 INFO:teuthology.orchestra.run.smithi143.stdout:ttyS1 2024-09-11T07:18:07.982 DEBUG:teuthology.parallel:result is None 2024-09-11T07:18:08.432 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-11T07:18:08.432 DEBUG:teuthology.orchestra.run.smithi164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T07:18:08.552 INFO:teuthology.orchestra.run.smithi164.stdout:ttyS1 2024-09-11T07:18:08.571 DEBUG:teuthology.parallel:result is None 2024-09-11T07:18:08.571 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T07:18:08.579 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T07:18:08.579 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T07:18:08.582 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T07:18:08.597 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T07:18:08.602 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T07:18:08.644 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T07:18:08.644 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T07:18:08.646 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T07:18:08.670 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T07:18:08.686 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T07:18:08.686 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T07:18:08.703 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T07:18:08.726 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:18:08.736 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:18:08.744 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:18:08.754 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:18:08.755 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T07:18:08.762 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T07:18:08.762 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T07:18:08.782 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T07:18:08.820 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T07:18:08.829 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T07:18:08.829 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T07:18:08.848 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T07:18:08.876 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:18:08.938 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:18:08.989 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:18:09.073 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:18:09.112 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-11T07:18:09.112 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T07:18:09.176 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:18:09.218 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:18:09.259 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:18:09.341 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:18:09.381 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-11T07:18:09.381 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T07:18:09.447 DEBUG:teuthology.orchestra.run.smithi143:> sudo service rsyslog restart 2024-09-11T07:18:09.449 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-09-11T07:18:09.491 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:18:09.548 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:18:09.838 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T07:18:09.845 INFO:teuthology.task.internal:Starting timer... 2024-09-11T07:18:09.846 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T07:18:09.881 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T07:18:09.914 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-11T07:18:09.914 DEBUG:teuthology.orchestra.run.smithi143:> sudo service auditd rotate 2024-09-11T07:18:09.973 INFO:teuthology.orchestra.run.smithi143.stdout:Rotating logs: 2024-09-11T07:18:09.974 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-09-11T07:18:10.040 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-09-11T07:18:10.042 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T07:18:10.042 DEBUG:teuthology.orchestra.run.smithi143:> /usr/sbin/getenforce 2024-09-11T07:18:10.065 INFO:teuthology.orchestra.run.smithi143.stdout:Permissive 2024-09-11T07:18:10.066 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-09-11T07:18:10.089 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-09-11T07:18:10.090 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi143.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi164.front.sepia.ceph.com': 'permissive'} 2024-09-11T07:18:10.090 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:18:10.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:18:10.130 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:18:10.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:18:10.156 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T07:18:10.157 DEBUG:teuthology.orchestra.run.smithi143:> sudo /usr/sbin/setenforce permissive 2024-09-11T07:18:10.279 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-09-11T07:18:10.314 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T07:18:10.324 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-11T07:18:10.328 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-11T07:18:10.458 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T07:18:10.469 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T07:18:10.470 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi143.front.sepia.ceph.com,smithi164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T07:21:52.205 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi143.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com')] 2024-09-11T07:21:52.206 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-09-11T07:21:52.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:21:52.285 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-09-11T07:21:52.364 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-09-11T07:21:52.365 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-11T07:21:52.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:21:52.457 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-11T07:21:52.545 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-11T07:21:52.545 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T07:21:52.556 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T07:21:52.556 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T07:21:52.557 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T07:21:52.559 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T07:21:52.559 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T07:21:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T07:21:52.601 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T07:21:52.615 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T07:21:52.627 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: ntpd: command not found 2024-09-11T07:21:52.630 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T07:21:52.638 INFO:teuthology.orchestra.run.smithi143.stdout:506 Cannot talk to daemon 2024-09-11T07:21:52.652 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T07:21:52.654 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-09-11T07:21:52.665 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-09-11T07:21:52.666 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T07:21:52.679 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T07:21:52.694 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T07:21:52.720 INFO:teuthology.orchestra.run.smithi143.stderr:bash: line 1: ntpq: command not found 2024-09-11T07:21:52.723 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T07:21:52.723 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2024-09-11T07:21:52.723 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:21:52.723 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:21:52.723 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:21:52.723 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:21:52.750 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-09-11T07:21:52.753 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T07:21:52.753 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-09-11T07:21:52.754 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:21:52.754 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:21:52.754 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:21:52.754 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:21:52.755 INFO:teuthology.run_tasks:Running task pexec... 2024-09-11T07:21:52.764 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-11T07:21:52.765 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi143.front.sepia.ceph.com 2024-09-11T07:21:52.765 DEBUG:teuthology.orchestra.run.smithi143:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T07:21:52.765 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi164.front.sepia.ceph.com 2024-09-11T07:21:52.765 DEBUG:teuthology.orchestra.run.smithi164:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T07:21:53.114 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:21:53.128 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:21:53.128 INFO:teuthology.orchestra.run.smithi143.stdout: Package Architecture Version Repository Size 2024-09-11T07:21:53.128 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:21:53.128 INFO:teuthology.orchestra.run.smithi143.stdout:Removing: 2024-09-11T07:21:53.128 INFO:teuthology.orchestra.run.smithi143.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T07:21:53.128 INFO:teuthology.orchestra.run.smithi143.stdout:Removing unused dependencies: 2024-09-11T07:21:53.128 INFO:teuthology.orchestra.run.smithi143.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T07:21:53.128 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:21:53.128 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:21:53.128 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:21:53.129 INFO:teuthology.orchestra.run.smithi143.stdout:Remove 2 Packages 2024-09-11T07:21:53.129 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:21:53.129 INFO:teuthology.orchestra.run.smithi143.stdout:Freed space: 5.5 M 2024-09-11T07:21:53.129 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:21:53.132 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:21:53.133 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:21:53.148 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:21:53.172 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:21:53.172 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-09-11T07:21:53.172 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:21:53.172 INFO:teuthology.orchestra.run.smithi164.stdout:Removing: 2024-09-11T07:21:53.172 INFO:teuthology.orchestra.run.smithi164.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T07:21:53.172 INFO:teuthology.orchestra.run.smithi164.stdout:Removing unused dependencies: 2024-09-11T07:21:53.173 INFO:teuthology.orchestra.run.smithi164.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T07:21:53.173 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:21:53.173 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:21:53.173 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:21:53.173 INFO:teuthology.orchestra.run.smithi164.stdout:Remove 2 Packages 2024-09-11T07:21:53.173 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:21:53.173 INFO:teuthology.orchestra.run.smithi164.stdout:Freed space: 5.5 M 2024-09-11T07:21:53.173 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:21:53.177 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:21:53.177 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:21:53.194 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:21:53.194 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:21:53.244 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:21:53.244 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:21:53.333 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:21:53.392 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:21:53.417 INFO:teuthology.orchestra.run.smithi143.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T07:21:53.465 INFO:teuthology.orchestra.run.smithi164.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T07:21:53.479 INFO:teuthology.orchestra.run.smithi143.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:21:53.533 INFO:teuthology.orchestra.run.smithi164.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:21:53.881 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:21:53.881 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T07:21:54.072 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:21:54.072 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T07:21:54.139 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T07:21:54.139 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:21:54.139 INFO:teuthology.orchestra.run.smithi143.stdout:Removed: 2024-09-11T07:21:54.139 INFO:teuthology.orchestra.run.smithi143.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:21:54.139 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:21:54.139 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:21:54.371 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T07:21:54.371 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:21:54.371 INFO:teuthology.orchestra.run.smithi164.stdout:Removed: 2024-09-11T07:21:54.371 INFO:teuthology.orchestra.run.smithi164.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:21:54.372 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:21:54.372 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:21:55.007 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:00:48 ago on Wed 11 Sep 2024 07:21:06 AM UTC. 2024-09-11T07:21:55.153 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:21:55.154 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:21:55.154 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-09-11T07:21:55.154 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:21:55.154 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:21:55.154 INFO:teuthology.orchestra.run.smithi143.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T07:21:55.154 INFO:teuthology.orchestra.run.smithi143.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T07:21:55.155 INFO:teuthology.orchestra.run.smithi143.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-11T07:21:55.155 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-09-11T07:21:55.155 INFO:teuthology.orchestra.run.smithi143.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T07:21:55.155 INFO:teuthology.orchestra.run.smithi143.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T07:21:55.155 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T07:21:55.155 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T07:21:55.155 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:21:55.155 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:21:55.155 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:21:55.155 INFO:teuthology.orchestra.run.smithi143.stdout:Install 7 Packages 2024-09-11T07:21:55.155 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:21:55.184 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 5.2 M 2024-09-11T07:21:55.184 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 19 M 2024-09-11T07:21:55.184 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:21:55.239 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:49 ago on Wed 11 Sep 2024 07:21:06 AM UTC. 2024-09-11T07:21:55.376 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:21:55.377 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:21:55.377 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-11T07:21:55.377 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:21:55.377 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:21:55.377 INFO:teuthology.orchestra.run.smithi164.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T07:21:55.377 INFO:teuthology.orchestra.run.smithi164.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T07:21:55.377 INFO:teuthology.orchestra.run.smithi164.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-11T07:21:55.377 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-11T07:21:55.377 INFO:teuthology.orchestra.run.smithi164.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T07:21:55.377 INFO:teuthology.orchestra.run.smithi164.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T07:21:55.377 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T07:21:55.378 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T07:21:55.378 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:21:55.378 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:21:55.378 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:21:55.378 INFO:teuthology.orchestra.run.smithi164.stdout:Install 7 Packages 2024-09-11T07:21:55.378 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:21:55.401 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 5.2 M 2024-09-11T07:21:55.401 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 19 M 2024-09-11T07:21:55.401 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:21:55.528 INFO:teuthology.orchestra.run.smithi143.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 402 kB/s | 44 kB 00:00 2024-09-11T07:21:55.553 INFO:teuthology.orchestra.run.smithi143.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 781 kB/s | 106 kB 00:00 2024-09-11T07:21:55.603 INFO:teuthology.orchestra.run.smithi143.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 954 kB/s | 72 kB 00:00 2024-09-11T07:21:55.629 INFO:teuthology.orchestra.run.smithi143.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.1 MB/s | 84 kB 00:00 2024-09-11T07:21:55.650 INFO:teuthology.orchestra.run.smithi164.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 344 kB/s | 44 kB 00:00 2024-09-11T07:21:55.684 INFO:teuthology.orchestra.run.smithi164.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 649 kB/s | 106 kB 00:00 2024-09-11T07:21:55.726 INFO:teuthology.orchestra.run.smithi164.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 953 kB/s | 72 kB 00:00 2024-09-11T07:21:55.752 INFO:teuthology.orchestra.run.smithi164.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.2 MB/s | 84 kB 00:00 2024-09-11T07:21:55.779 INFO:teuthology.orchestra.run.smithi143.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.4 MB/s | 906 kB 00:00 2024-09-11T07:21:55.846 INFO:teuthology.orchestra.run.smithi143.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 3.4 MB/s | 837 kB 00:00 2024-09-11T07:21:55.868 INFO:teuthology.orchestra.run.smithi164.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.5 MB/s | 906 kB 00:00 2024-09-11T07:21:55.952 INFO:teuthology.orchestra.run.smithi164.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 3.6 MB/s | 837 kB 00:00 2024-09-11T07:21:56.269 INFO:teuthology.orchestra.run.smithi164.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 6.1 MB/s | 3.2 MB 00:00 2024-09-11T07:21:56.269 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:21:56.269 INFO:teuthology.orchestra.run.smithi164.stdout:Total 6.0 MB/s | 5.2 MB 00:00 2024-09-11T07:21:56.330 INFO:teuthology.orchestra.run.smithi143.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 4.5 MB/s | 3.2 MB 00:00 2024-09-11T07:21:56.330 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:21:56.330 INFO:teuthology.orchestra.run.smithi143.stdout:Total 4.5 MB/s | 5.2 MB 00:01 2024-09-11T07:21:56.379 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:21:56.394 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:21:56.394 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:21:56.437 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:21:56.451 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:21:56.451 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:21:56.693 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:21:56.693 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:21:56.762 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:21:56.763 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:21:57.201 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:21:57.236 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:21:57.288 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-11T07:21:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-11T07:21:57.324 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-11T07:21:57.351 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-11T07:21:57.358 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-11T07:21:57.387 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-11T07:21:57.572 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-11T07:21:57.602 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-11T07:21:57.607 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T07:21:57.627 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T07:21:58.043 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T07:21:58.061 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T07:21:58.069 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T07:21:58.088 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T07:21:58.200 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T07:21:58.202 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T07:21:58.266 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T07:21:58.272 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T07:21:59.263 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T07:21:59.264 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-11T07:21:59.264 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-11T07:21:59.264 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-11T07:21:59.264 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-11T07:21:59.264 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-11T07:21:59.264 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-11T07:21:59.307 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T07:21:59.307 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-11T07:21:59.307 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-11T07:21:59.308 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-11T07:21:59.308 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-11T07:21:59.308 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-11T07:21:59.308 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-11T07:21:59.586 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T07:21:59.586 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:21:59.586 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:21:59.586 INFO:teuthology.orchestra.run.smithi164.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:21:59.586 INFO:teuthology.orchestra.run.smithi164.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T07:21:59.586 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T07:21:59.586 INFO:teuthology.orchestra.run.smithi164.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-11T07:21:59.586 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:21:59.586 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:21:59.641 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T07:21:59.641 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:21:59.641 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:21:59.641 INFO:teuthology.orchestra.run.smithi143.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:21:59.641 INFO:teuthology.orchestra.run.smithi143.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T07:21:59.641 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T07:21:59.641 INFO:teuthology.orchestra.run.smithi143.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-11T07:21:59.642 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:21:59.642 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:21:59.976 DEBUG:teuthology.parallel:result is None 2024-09-11T07:21:59.985 DEBUG:teuthology.parallel:result is None 2024-09-11T07:21:59.985 INFO:teuthology.run_tasks:Running task install... 2024-09-11T07:21:59.994 DEBUG:teuthology.task.install:project ceph 2024-09-11T07:21:59.994 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'}, 'extra_packages': ['cephadm']} 2024-09-11T07:21:59.994 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'} 2024-09-11T07:21:59.994 INFO:teuthology.task.install:Using flavor: default 2024-09-11T07:22:00.000 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T07:22:00.000 INFO:teuthology.task.install:extra packages: [] 2024-09-11T07:22:00.000 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'tag': None, 'wait_for_package': False} 2024-09-11T07:22:00.001 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:22:00.003 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'tag': None, 'wait_for_package': False} 2024-09-11T07:22:00.004 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:22:00.150 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/ 2024-09-11T07:22:00.151 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1623.gd0e6828a 2024-09-11T07:22:00.159 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/ 2024-09-11T07:22:00.159 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1623.gd0e6828a 2024-09-11T07:22:00.265 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T07:22:00.265 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-11T07:22:00.265 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T07:22:00.270 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T07:22:00.270 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-11T07:22:00.270 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T07:22:00.291 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T07:22:00.292 DEBUG:teuthology.orchestra.run.smithi143:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e6828a2016d48cf25ad84064e50742bb1c39b9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T07:22:00.296 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T07:22:00.296 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e6828a2016d48cf25ad84064e50742bb1c39b9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T07:22:00.358 DEBUG:teuthology.orchestra.run.smithi143:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T07:22:00.361 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T07:22:00.437 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T07:22:00.438 DEBUG:teuthology.orchestra.run.smithi143:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T07:22:00.501 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2024-09-11T07:22:00.503 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2024-09-11T07:22:00.504 INFO:teuthology.orchestra.run.smithi143.stdout:check_obsoletes = 1 2024-09-11T07:22:00.504 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum clean all 2024-09-11T07:22:00.735 INFO:teuthology.orchestra.run.smithi143.stdout:45 files removed 2024-09-11T07:22:00.749 INFO:teuthology.orchestra.run.smithi164.stdout:45 files removed 2024-09-11T07:22:00.761 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-radosgw 2024-09-11T07:22:00.777 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw 2024-09-11T07:22:01.321 INFO:teuthology.orchestra.run.smithi143.stdout:ceph packages for x86_64 284 kB/s | 77 kB 00:00 2024-09-11T07:22:01.335 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 283 kB/s | 77 kB 00:00 2024-09-11T07:22:01.605 INFO:teuthology.orchestra.run.smithi143.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-11T07:22:01.619 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-11T07:22:01.890 INFO:teuthology.orchestra.run.smithi143.stdout:ceph source packages 7.2 kB/s | 1.8 kB 00:00 2024-09-11T07:22:01.894 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 7.5 kB/s | 1.8 kB 00:00 2024-09-11T07:22:02.478 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-11T07:22:02.681 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-11T07:22:06.899 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - AppStream 6.5 MB/s | 20 MB 00:03 2024-09-11T07:22:09.154 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 3.7 MB/s | 20 MB 00:05 2024-09-11T07:22:11.783 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - CRB 5.9 MB/s | 6.5 MB 00:01 2024-09-11T07:22:13.684 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 19 kB 00:00 2024-09-11T07:22:14.454 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.5 MB 00:01 2024-09-11T07:22:14.463 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-11T07:22:15.905 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 19 kB 00:00 2024-09-11T07:22:17.143 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-11T07:22:19.894 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-11T07:22:22.237 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:22:22.239 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:22:22.239 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-09-11T07:22:22.239 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:22:22.240 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:22:22.240 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 14 M 2024-09-11T07:22:22.240 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrading: 2024-09-11T07:22:22.240 INFO:teuthology.orchestra.run.smithi143.stdout: librados2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.1 M 2024-09-11T07:22:22.240 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:22:22.240 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-09-11T07:22:22.240 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 5.0 M 2024-09-11T07:22:22.240 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 23 M 2024-09-11T07:22:22.240 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:22:22.240 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T07:22:22.241 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 636 k 2024-09-11T07:22:22.241 INFO:teuthology.orchestra.run.smithi143.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T07:22:22.241 INFO:teuthology.orchestra.run.smithi143.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T07:22:22.241 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T07:22:22.241 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 457 k 2024-09-11T07:22:22.241 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T07:22:22.241 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 4.9 M 2024-09-11T07:22:22.241 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T07:22:22.241 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T07:22:22.241 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 45 k 2024-09-11T07:22:22.241 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 120 k 2024-09-11T07:22:22.242 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 156 k 2024-09-11T07:22:22.242 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T07:22:22.242 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 320 k 2024-09-11T07:22:22.242 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 294 k 2024-09-11T07:22:22.242 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 99 k 2024-09-11T07:22:22.242 INFO:teuthology.orchestra.run.smithi143.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T07:22:22.242 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:22:22.242 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:22:22.242 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:22:22.242 INFO:teuthology.orchestra.run.smithi143.stdout:Install 22 Packages 2024-09-11T07:22:22.242 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrade 2 Packages 2024-09-11T07:22:22.243 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:22:22.243 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 58 M 2024-09-11T07:22:22.243 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:22:22.740 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-11T07:22:22.870 INFO:teuthology.orchestra.run.smithi143.stdout:(1/24): ceph-base-17.2.7-1623.gd0e6828a.el9.x86 9.6 MB/s | 5.0 MB 00:00 2024-09-11T07:22:22.945 INFO:teuthology.orchestra.run.smithi143.stdout:(2/24): ceph-selinux-17.2.7-1623.gd0e6828a.el9. 336 kB/s | 25 kB 00:00 2024-09-11T07:22:23.046 INFO:teuthology.orchestra.run.smithi143.stdout:(3/24): libcephfs2-17.2.7-1623.gd0e6828a.el9.x8 6.2 MB/s | 636 kB 00:00 2024-09-11T07:22:23.204 INFO:teuthology.orchestra.run.smithi143.stdout:(4/24): ceph-radosgw-17.2.7-1623.gd0e6828a.el9. 17 MB/s | 14 MB 00:00 2024-09-11T07:22:23.230 INFO:teuthology.orchestra.run.smithi143.stdout:(5/24): libradosstriper1-17.2.7-1623.gd0e6828a. 2.4 MB/s | 457 kB 00:00 2024-09-11T07:22:23.297 INFO:teuthology.orchestra.run.smithi143.stdout:(6/24): python3-ceph-argparse-17.2.7-1623.gd0e6 677 kB/s | 45 kB 00:00 2024-09-11T07:22:23.414 INFO:teuthology.orchestra.run.smithi143.stdout:(7/24): librgw2-17.2.7-1623.gd0e6828a.el9.x86_6 24 MB/s | 4.9 MB 00:00 2024-09-11T07:22:23.439 INFO:teuthology.orchestra.run.smithi143.stdout:(8/24): python3-ceph-common-17.2.7-1623.gd0e682 846 kB/s | 120 kB 00:00 2024-09-11T07:22:23.473 INFO:teuthology.orchestra.run.smithi143.stdout:(9/24): python3-cephfs-17.2.7-1623.gd0e6828a.el 2.6 MB/s | 156 kB 00:00 2024-09-11T07:22:23.506 INFO:teuthology.orchestra.run.smithi143.stdout:(10/24): python3-rados-17.2.7-1623.gd0e6828a.el 4.7 MB/s | 320 kB 00:00 2024-09-11T07:22:23.532 INFO:teuthology.orchestra.run.smithi143.stdout:(11/24): python3-rbd-17.2.7-1623.gd0e6828a.el9. 4.9 MB/s | 294 kB 00:00 2024-09-11T07:22:23.565 INFO:teuthology.orchestra.run.smithi143.stdout:(12/24): python3-rgw-17.2.7-1623.gd0e6828a.el9. 1.7 MB/s | 99 kB 00:00 2024-09-11T07:22:23.841 INFO:teuthology.orchestra.run.smithi143.stdout:(13/24): ceph-common-17.2.7-1623.gd0e6828a.el9. 15 MB/s | 23 MB 00:01 2024-09-11T07:22:23.866 INFO:teuthology.orchestra.run.smithi143.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 151 kB/s | 45 kB 00:00 2024-09-11T07:22:23.899 INFO:teuthology.orchestra.run.smithi143.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 436 kB/s | 160 kB 00:00 2024-09-11T07:22:23.941 INFO:teuthology.orchestra.run.smithi143.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-11T07:22:23.975 INFO:teuthology.orchestra.run.smithi143.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-11T07:22:24.026 INFO:teuthology.orchestra.run.smithi143.stdout:(18/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 3.6 MB/s | 309 kB 00:00 2024-09-11T07:22:24.067 INFO:teuthology.orchestra.run.smithi143.stdout:(19/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-09-11T07:22:24.092 INFO:teuthology.orchestra.run.smithi143.stdout:(20/24): liboath-2.6.7-2.el9.x86_64.rpm 442 kB/s | 49 kB 00:00 2024-09-11T07:22:24.126 INFO:teuthology.orchestra.run.smithi143.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 683 kB/s | 67 kB 00:00 2024-09-11T07:22:24.185 INFO:teuthology.orchestra.run.smithi143.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-11T07:22:24.318 INFO:teuthology.orchestra.run.smithi143.stdout:(23/24): librados2-17.2.7-1623.gd0e6828a.el9.x8 14 MB/s | 3.1 MB 00:00 2024-09-11T07:22:24.369 INFO:teuthology.orchestra.run.smithi143.stdout:(24/24): librbd1-17.2.7-1623.gd0e6828a.el9.x86_ 12 MB/s | 2.9 MB 00:00 2024-09-11T07:22:24.374 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:22:24.374 INFO:teuthology.orchestra.run.smithi143.stdout:Total 27 MB/s | 58 MB 00:02 2024-09-11T07:22:24.438 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:22:24.472 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:22:24.472 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:22:24.940 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:22:24.940 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:22:25.093 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:22:25.094 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:22:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-11T07:22:25.095 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:22:25.095 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:22:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 14 M 2024-09-11T07:22:25.095 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2024-09-11T07:22:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.1 M 2024-09-11T07:22:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:22:25.095 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-11T07:22:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 5.0 M 2024-09-11T07:22:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 23 M 2024-09-11T07:22:25.095 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:22:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T07:22:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 636 k 2024-09-11T07:22:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T07:22:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T07:22:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T07:22:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 457 k 2024-09-11T07:22:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T07:22:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 4.9 M 2024-09-11T07:22:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T07:22:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T07:22:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 45 k 2024-09-11T07:22:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 120 k 2024-09-11T07:22:25.096 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 156 k 2024-09-11T07:22:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T07:22:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 320 k 2024-09-11T07:22:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 294 k 2024-09-11T07:22:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 99 k 2024-09-11T07:22:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T07:22:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:22:25.097 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:22:25.097 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:22:25.097 INFO:teuthology.orchestra.run.smithi164.stdout:Install 22 Packages 2024-09-11T07:22:25.097 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 2 Packages 2024-09-11T07:22:25.097 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:22:25.098 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 58 M 2024-09-11T07:22:25.098 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:22:25.891 INFO:teuthology.orchestra.run.smithi164.stdout:(1/24): ceph-base-17.2.7-1623.gd0e6828a.el9.x86 10 MB/s | 5.0 MB 00:00 2024-09-11T07:22:25.967 INFO:teuthology.orchestra.run.smithi164.stdout:(2/24): ceph-selinux-17.2.7-1623.gd0e6828a.el9. 335 kB/s | 25 kB 00:00 2024-09-11T07:22:26.075 INFO:teuthology.orchestra.run.smithi164.stdout:(3/24): libcephfs2-17.2.7-1623.gd0e6828a.el9.x8 5.7 MB/s | 636 kB 00:00 2024-09-11T07:22:26.135 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:22:26.184 INFO:teuthology.orchestra.run.smithi164.stdout:(4/24): libradosstriper1-17.2.7-1623.gd0e6828a. 4.1 MB/s | 457 kB 00:00 2024-09-11T07:22:26.393 INFO:teuthology.orchestra.run.smithi164.stdout:(5/24): ceph-radosgw-17.2.7-1623.gd0e6828a.el9. 15 MB/s | 14 MB 00:00 2024-09-11T07:22:26.434 INFO:teuthology.orchestra.run.smithi164.stdout:(6/24): python3-ceph-argparse-17.2.7-1623.gd0e6 1.1 MB/s | 45 kB 00:00 2024-09-11T07:22:26.478 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-11T07:22:26.526 INFO:teuthology.orchestra.run.smithi164.stdout:(7/24): librgw2-17.2.7-1623.gd0e6828a.el9.x86_6 14 MB/s | 4.9 MB 00:00 2024-09-11T07:22:26.579 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:22:26.671 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:22:26.743 INFO:teuthology.orchestra.run.smithi164.stdout:(8/24): ceph-common-17.2.7-1623.gd0e6828a.el9.x 17 MB/s | 23 MB 00:01 2024-09-11T07:22:26.749 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T07:22:26.769 INFO:teuthology.orchestra.run.smithi164.stdout:(9/24): python3-ceph-common-17.2.7-1623.gd0e682 360 kB/s | 120 kB 00:00 2024-09-11T07:22:26.789 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:22:26.811 INFO:teuthology.orchestra.run.smithi164.stdout:(10/24): python3-cephfs-17.2.7-1623.gd0e6828a.e 550 kB/s | 156 kB 00:00 2024-09-11T07:22:26.836 INFO:teuthology.orchestra.run.smithi164.stdout:(11/24): python3-rados-17.2.7-1623.gd0e6828a.el 3.4 MB/s | 320 kB 00:00 2024-09-11T07:22:26.860 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:22:26.861 INFO:teuthology.orchestra.run.smithi164.stdout:(12/24): python3-rbd-17.2.7-1623.gd0e6828a.el9. 3.1 MB/s | 294 kB 00:00 2024-09-11T07:22:26.887 INFO:teuthology.orchestra.run.smithi164.stdout:(13/24): python3-rgw-17.2.7-1623.gd0e6828a.el9. 1.3 MB/s | 99 kB 00:00 2024-09-11T07:22:26.915 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-11T07:22:26.953 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-11T07:22:26.995 INFO:teuthology.orchestra.run.smithi164.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 418 kB/s | 45 kB 00:00 2024-09-11T07:22:27.021 INFO:teuthology.orchestra.run.smithi164.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 870 kB/s | 160 kB 00:00 2024-09-11T07:22:27.071 INFO:teuthology.orchestra.run.smithi164.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 843 kB/s | 42 kB 00:00 2024-09-11T07:22:27.105 INFO:teuthology.orchestra.run.smithi164.stdout:(17/24): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-09-11T07:22:27.138 INFO:teuthology.orchestra.run.smithi164.stdout:(18/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-09-11T07:22:27.181 INFO:teuthology.orchestra.run.smithi164.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 713 kB/s | 49 kB 00:00 2024-09-11T07:22:27.205 INFO:teuthology.orchestra.run.smithi164.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.2 MB/s | 309 kB 00:00 2024-09-11T07:22:27.231 INFO:teuthology.orchestra.run.smithi164.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 809 kB/s | 67 kB 00:00 2024-09-11T07:22:27.281 INFO:teuthology.orchestra.run.smithi164.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-11T07:22:27.392 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-11T07:22:27.418 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T07:22:27.465 INFO:teuthology.orchestra.run.smithi164.stdout:(23/24): librados2-17.2.7-1623.gd0e6828a.el9.x8 12 MB/s | 3.1 MB 00:00 2024-09-11T07:22:27.480 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T07:22:27.521 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-11T07:22:27.531 INFO:teuthology.orchestra.run.smithi164.stdout:(24/24): librbd1-17.2.7-1623.gd0e6828a.el9.x86_ 9.8 MB/s | 2.9 MB 00:00 2024-09-11T07:22:27.537 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:22:27.537 INFO:teuthology.orchestra.run.smithi164.stdout:Total 24 MB/s | 58 MB 00:02 2024-09-11T07:22:27.572 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 10/26 2024-09-11T07:22:27.601 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:22:27.609 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T07:22:27.636 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:22:27.636 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:22:27.672 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T07:22:27.699 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T07:22:27.765 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T07:22:27.805 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-11T07:22:27.841 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-11T07:22:27.874 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-11T07:22:28.098 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:22:28.098 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:22:28.168 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-11T07:22:28.212 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T07:22:28.319 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T07:22:28.391 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 19/26 2024-09-11T07:22:28.423 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 20/26 2024-09-11T07:22:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:22:29.651 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-11T07:22:29.734 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:22:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:22:29.903 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T07:22:29.960 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:22:30.047 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:22:30.114 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-11T07:22:30.157 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-11T07:22:30.612 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-11T07:22:30.663 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T07:22:30.726 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T07:22:30.758 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-11T07:22:30.800 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 10/26 2024-09-11T07:22:30.837 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T07:22:30.871 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:22:30.908 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T07:22:30.911 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:22:30.944 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T07:22:31.016 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T07:22:31.059 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-11T07:22:31.094 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-11T07:22:31.126 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-11T07:22:31.429 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-11T07:22:31.440 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:22:31.465 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T07:22:31.482 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T07:22:31.555 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T07:22:31.565 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T07:22:31.565 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T07:22:31.565 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T07:22:31.566 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:22:31.608 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:22:31.629 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 19/26 2024-09-11T07:22:31.659 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 20/26 2024-09-11T07:22:34.085 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:22:34.123 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:22:34.636 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:22:34.668 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T07:22:34.840 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T07:22:34.840 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T07:22:34.840 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T07:22:34.840 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:22:34.869 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:22:41.665 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:22:41.665 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /sys 2024-09-11T07:22:41.665 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /proc 2024-09-11T07:22:41.665 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /mnt 2024-09-11T07:22:41.665 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /var/tmp 2024-09-11T07:22:41.665 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /home 2024-09-11T07:22:41.665 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /root 2024-09-11T07:22:41.665 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /tmp 2024-09-11T07:22:41.666 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:22:43.002 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T07:22:43.089 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T07:22:43.089 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:22:43.089 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T07:22:43.089 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:22:43.089 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:22:43.089 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:22:43.091 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T07:22:43.172 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T07:22:43.172 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:22:44.525 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:22:44.526 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/26 2024-09-11T07:22:44.526 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:22:44.526 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T07:22:44.526 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:22:44.526 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/26 2024-09-11T07:22:44.526 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 6/26 2024-09-11T07:22:44.526 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/26 2024-09-11T07:22:44.526 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 8/26 2024-09-11T07:22:44.526 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 9/26 2024-09-11T07:22:44.527 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/26 2024-09-11T07:22:44.527 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T07:22:44.527 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T07:22:44.527 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 13/26 2024-09-11T07:22:44.527 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-11T07:22:44.527 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-11T07:22:44.527 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-11T07:22:44.527 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-11T07:22:44.527 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-11T07:22:44.527 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-11T07:22:44.527 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-11T07:22:44.527 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-11T07:22:44.527 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-11T07:22:44.528 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:22:44.528 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-11T07:22:44.528 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 25/26 2024-09-11T07:22:44.811 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:22:44.812 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2024-09-11T07:22:44.812 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2024-09-11T07:22:44.812 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2024-09-11T07:22:44.812 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2024-09-11T07:22:44.812 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2024-09-11T07:22:44.812 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /root 2024-09-11T07:22:44.812 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2024-09-11T07:22:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:22:45.550 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:22:45.550 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:22:45.550 INFO:teuthology.orchestra.run.smithi143.stdout:Upgraded: 2024-09-11T07:22:45.550 INFO:teuthology.orchestra.run.smithi143.stdout: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:45.550 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:45.550 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:22:45.550 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:45.550 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:45.551 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:45.551 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:45.551 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T07:22:45.551 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:45.551 INFO:teuthology.orchestra.run.smithi143.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T07:22:45.551 INFO:teuthology.orchestra.run.smithi143.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T07:22:45.551 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T07:22:45.551 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:45.551 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T07:22:45.551 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:45.551 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T07:22:45.551 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T07:22:45.552 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:45.552 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:45.552 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:45.552 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T07:22:45.552 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:45.552 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:45.552 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:45.552 INFO:teuthology.orchestra.run.smithi143.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T07:22:45.552 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:22:45.552 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:22:45.915 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-test 2024-09-11T07:22:46.110 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T07:22:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T07:22:46.186 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:22:46.186 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T07:22:46.186 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:22:46.186 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:22:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:22:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T07:22:46.279 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T07:22:46.280 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:22:46.486 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:00:27 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:22:46.621 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:22:46.622 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:22:46.622 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-09-11T07:22:46.622 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:22:46.622 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:22:46.622 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 48 M 2024-09-11T07:22:46.622 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-09-11T07:22:46.622 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 163 k 2024-09-11T07:22:46.622 INFO:teuthology.orchestra.run.smithi143.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T07:22:46.622 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T07:22:46.622 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:22:46.622 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:22:46.622 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:22:46.623 INFO:teuthology.orchestra.run.smithi143.stdout:Install 4 Packages 2024-09-11T07:22:46.623 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:22:46.623 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 48 M 2024-09-11T07:22:46.623 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 193 M 2024-09-11T07:22:46.623 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:22:46.860 INFO:teuthology.orchestra.run.smithi143.stdout:(1/4): libcephsqlite-17.2.7-1623.gd0e6828a.el9. 861 kB/s | 163 kB 00:00 2024-09-11T07:22:47.468 INFO:teuthology.orchestra.run.smithi143.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 105 kB/s | 64 kB 00:00 2024-09-11T07:22:47.541 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:22:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/26 2024-09-11T07:22:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:22:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T07:22:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:22:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/26 2024-09-11T07:22:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 6/26 2024-09-11T07:22:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/26 2024-09-11T07:22:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 8/26 2024-09-11T07:22:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 9/26 2024-09-11T07:22:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/26 2024-09-11T07:22:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T07:22:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T07:22:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 13/26 2024-09-11T07:22:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-11T07:22:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-11T07:22:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-11T07:22:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-11T07:22:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-11T07:22:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-11T07:22:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-11T07:22:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-11T07:22:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-11T07:22:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:22:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-11T07:22:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 25/26 2024-09-11T07:22:47.610 INFO:teuthology.orchestra.run.smithi143.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 324 kB/s | 304 kB 00:00 2024-09-11T07:22:48.177 INFO:teuthology.orchestra.run.smithi143.stdout:(4/4): ceph-test-17.2.7-1623.gd0e6828a.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-11T07:22:48.179 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:22:48.179 INFO:teuthology.orchestra.run.smithi143.stdout:Total 31 MB/s | 48 MB 00:01 2024-09-11T07:22:48.207 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:22:48.227 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:22:48.227 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:22:48.449 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:22:48.449 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:22:48.449 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2024-09-11T07:22:48.449 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:48.450 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:48.450 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:22:48.450 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:48.450 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:48.450 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:48.450 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:48.450 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T07:22:48.450 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:48.450 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T07:22:48.450 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T07:22:48.450 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T07:22:48.450 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:48.450 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T07:22:48.451 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:48.451 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T07:22:48.451 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T07:22:48.451 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:48.451 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:48.451 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:48.451 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T07:22:48.451 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:48.451 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:48.451 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:48.451 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T07:22:48.451 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:22:48.451 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:22:48.596 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:22:48.597 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:22:48.810 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-test 2024-09-11T07:22:49.114 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:22:49.185 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T07:22:49.229 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T07:22:49.257 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T07:22:49.386 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:27 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:22:49.520 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:22:49.520 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:22:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-11T07:22:49.521 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:22:49.521 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:22:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 48 M 2024-09-11T07:22:49.521 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-11T07:22:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 163 k 2024-09-11T07:22:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T07:22:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T07:22:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:22:49.521 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:22:49.521 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:22:49.521 INFO:teuthology.orchestra.run.smithi164.stdout:Install 4 Packages 2024-09-11T07:22:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:22:49.522 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 48 M 2024-09-11T07:22:49.522 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 193 M 2024-09-11T07:22:49.522 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:22:49.903 INFO:teuthology.orchestra.run.smithi164.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-11T07:22:49.945 INFO:teuthology.orchestra.run.smithi164.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-11T07:22:49.999 INFO:teuthology.orchestra.run.smithi164.stdout:(3/4): libcephsqlite-17.2.7-1623.gd0e6828a.el9. 739 kB/s | 163 kB 00:00 2024-09-11T07:22:51.282 INFO:teuthology.orchestra.run.smithi164.stdout:(4/4): ceph-test-17.2.7-1623.gd0e6828a.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-11T07:22:51.285 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:22:51.285 INFO:teuthology.orchestra.run.smithi164.stdout:Total 27 MB/s | 48 MB 00:01 2024-09-11T07:22:51.314 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:22:51.333 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:22:51.333 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:22:51.704 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:22:51.704 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:22:52.239 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:22:52.337 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T07:22:52.390 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T07:22:52.418 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T07:22:53.706 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T07:22:53.781 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T07:22:56.061 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T07:22:56.062 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/4 2024-09-11T07:22:56.062 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/4 2024-09-11T07:22:56.062 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T07:22:56.360 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T07:22:56.360 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:22:56.360 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:22:56.360 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:56.360 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:56.360 INFO:teuthology.orchestra.run.smithi143.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T07:22:56.360 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T07:22:56.360 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:22:56.360 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:22:56.660 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph 2024-09-11T07:22:56.973 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T07:22:57.067 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T07:22:57.215 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:00:38 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:22:57.351 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:22:57.353 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:22:57.353 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-09-11T07:22:57.353 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:22:57.353 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:22:57.353 INFO:teuthology.orchestra.run.smithi143.stdout: ceph x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 6.5 k 2024-09-11T07:22:57.353 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-09-11T07:22:57.353 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.1 M 2024-09-11T07:22:57.353 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 1.4 M 2024-09-11T07:22:57.354 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 240 k 2024-09-11T07:22:57.354 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.9 M 2024-09-11T07:22:57.354 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 15 M 2024-09-11T07:22:57.354 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T07:22:57.354 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T07:22:57.354 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T07:22:57.354 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T07:22:57.354 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T07:22:57.354 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T07:22:57.354 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T07:22:57.354 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-collections 2024-09-11T07:22:57.354 INFO:teuthology.orchestra.run.smithi143.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-11T07:22:57.354 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T07:22:57.355 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T07:22:57.355 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T07:22:57.355 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T07:22:57.355 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T07:22:57.355 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T07:22:57.355 INFO:teuthology.orchestra.run.smithi143.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T07:22:57.355 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T07:22:57.355 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T07:22:57.355 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T07:22:57.355 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T07:22:57.356 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T07:22:57.356 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T07:22:57.356 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T07:22:57.356 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T07:22:57.356 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T07:22:57.356 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T07:22:57.356 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T07:22:57.356 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T07:22:57.356 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T07:22:57.356 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:22:57.356 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:22:57.357 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:22:57.357 INFO:teuthology.orchestra.run.smithi143.stdout:Install 34 Packages 2024-09-11T07:22:57.357 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:22:57.357 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 27 M 2024-09-11T07:22:57.357 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 96 M 2024-09-11T07:22:57.357 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:22:57.674 INFO:teuthology.orchestra.run.smithi143.stdout:(1/34): ceph-17.2.7-1623.gd0e6828a.el9.x86_64.r 58 kB/s | 6.5 kB 00:00 2024-09-11T07:22:57.883 INFO:teuthology.orchestra.run.smithi143.stdout:(2/34): ceph-mds-17.2.7-1623.gd0e6828a.el9.x86_ 6.4 MB/s | 2.1 MB 00:00 2024-09-11T07:22:57.912 INFO:teuthology.orchestra.run.smithi143.stdout:(3/34): ceph-mgr-17.2.7-1623.gd0e6828a.el9.x86_ 4.0 MB/s | 1.4 MB 00:00 2024-09-11T07:22:58.004 INFO:teuthology.orchestra.run.smithi143.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1623.gd0e6 2.6 MB/s | 240 kB 00:00 2024-09-11T07:22:58.071 INFO:teuthology.orchestra.run.smithi143.stdout:(5/34): ceph-mon-17.2.7-1623.gd0e6828a.el9.x86_ 9.9 MB/s | 3.9 MB 00:00 2024-09-11T07:22:58.188 INFO:teuthology.orchestra.run.smithi143.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-11T07:22:58.364 INFO:teuthology.orchestra.run.smithi143.stdout:(7/34): ceph-osd-17.2.7-1623.gd0e6828a.el9.x86_ 32 MB/s | 15 MB 00:00 2024-09-11T07:22:58.389 INFO:teuthology.orchestra.run.smithi143.stdout:(8/34): python3-ply-3.11-14.el9.noarch.rpm 531 kB/s | 106 kB 00:00 2024-09-11T07:22:58.431 INFO:teuthology.orchestra.run.smithi143.stdout:(9/34): python3-cryptography-36.0.1-4.el9.x86_6 3.4 MB/s | 1.2 MB 00:00 2024-09-11T07:22:58.456 INFO:teuthology.orchestra.run.smithi143.stdout:(10/34): python3-requests-2.25.1-8.el9.noarch.r 1.8 MB/s | 125 kB 00:00 2024-09-11T07:22:58.498 INFO:teuthology.orchestra.run.smithi143.stdout:(11/34): python3-pycparser-2.20-6.el9.noarch.rp 1.0 MB/s | 135 kB 00:00 2024-09-11T07:22:58.524 INFO:teuthology.orchestra.run.smithi143.stdout:(12/34): python3-urllib3-1.26.5-6.el9.noarch.rp 2.3 MB/s | 215 kB 00:00 2024-09-11T07:22:58.615 INFO:teuthology.orchestra.run.smithi143.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 297 kB/s | 35 kB 00:00 2024-09-11T07:22:58.674 INFO:teuthology.orchestra.run.smithi143.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 791 kB/s | 172 kB 00:00 2024-09-11T07:22:58.699 INFO:teuthology.orchestra.run.smithi143.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 440 kB/s | 77 kB 00:00 2024-09-11T07:22:58.725 INFO:teuthology.orchestra.run.smithi143.stdout:(16/34): python3-pytz-2021.1-5.el9.noarch.rpm 468 kB/s | 51 kB 00:00 2024-09-11T07:22:58.750 INFO:teuthology.orchestra.run.smithi143.stdout:(17/34): python3-toml-0.10.2-6.el9.noarch.rpm 551 kB/s | 42 kB 00:00 2024-09-11T07:22:58.808 INFO:teuthology.orchestra.run.smithi143.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 399 kB/s | 43 kB 00:00 2024-09-11T07:22:58.834 INFO:teuthology.orchestra.run.smithi143.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 464 kB/s | 11 kB 00:00 2024-09-11T07:22:58.860 INFO:teuthology.orchestra.run.smithi143.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-11T07:22:58.884 INFO:teuthology.orchestra.run.smithi143.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-11T07:22:58.909 INFO:teuthology.orchestra.run.smithi143.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 933 kB/s | 23 kB 00:00 2024-09-11T07:22:58.935 INFO:teuthology.orchestra.run.smithi143.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-09-11T07:22:58.960 INFO:teuthology.orchestra.run.smithi143.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 792 kB/s | 20 kB 00:00 2024-09-11T07:22:58.985 INFO:teuthology.orchestra.run.smithi143.stdout:(25/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.6 MB/s | 358 kB 00:00 2024-09-11T07:22:59.010 INFO:teuthology.orchestra.run.smithi143.stdout:(26/34): python3-logutils-0.3.5-21.el9.noarch.r 614 kB/s | 46 kB 00:00 2024-09-11T07:22:59.035 INFO:teuthology.orchestra.run.smithi143.stdout:(27/34): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-11T07:22:59.061 INFO:teuthology.orchestra.run.smithi143.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-11T07:22:59.086 INFO:teuthology.orchestra.run.smithi143.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-11T07:22:59.111 INFO:teuthology.orchestra.run.smithi143.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2024-09-11T07:22:59.137 INFO:teuthology.orchestra.run.smithi143.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-11T07:22:59.162 INFO:teuthology.orchestra.run.smithi143.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 804 kB/s | 20 kB 00:00 2024-09-11T07:22:59.187 INFO:teuthology.orchestra.run.smithi143.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-11T07:22:59.212 INFO:teuthology.orchestra.run.smithi143.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-11T07:22:59.216 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:22:59.217 INFO:teuthology.orchestra.run.smithi143.stdout:Total 15 MB/s | 27 MB 00:01 2024-09-11T07:22:59.317 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T07:22:59.317 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/4 2024-09-11T07:22:59.317 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/4 2024-09-11T07:22:59.317 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T07:22:59.341 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:22:59.377 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:22:59.377 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:22:59.668 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T07:22:59.668 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:22:59.668 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:22:59.668 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:59.669 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:22:59.669 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T07:22:59.669 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T07:22:59.669 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:22:59.669 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:22:59.764 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:22:59.765 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:22:59.925 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph 2024-09-11T07:23:00.504 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:38 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:00.641 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:00.644 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-11T07:23:00.644 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:00.644 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:23:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 6.5 k 2024-09-11T07:23:00.644 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-11T07:23:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.1 M 2024-09-11T07:23:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 1.4 M 2024-09-11T07:23:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 240 k 2024-09-11T07:23:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.9 M 2024-09-11T07:23:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 15 M 2024-09-11T07:23:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T07:23:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T07:23:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T07:23:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T07:23:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T07:23:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T07:23:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T07:23:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections 2024-09-11T07:23:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-11T07:23:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T07:23:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T07:23:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T07:23:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T07:23:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T07:23:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T07:23:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T07:23:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T07:23:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T07:23:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T07:23:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T07:23:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T07:23:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T07:23:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T07:23:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T07:23:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T07:23:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T07:23:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T07:23:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T07:23:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T07:23:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:00.647 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:23:00.648 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:00.648 INFO:teuthology.orchestra.run.smithi164.stdout:Install 34 Packages 2024-09-11T07:23:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:00.648 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 27 M 2024-09-11T07:23:00.648 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 96 M 2024-09-11T07:23:00.649 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:23:01.033 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:23:01.062 INFO:teuthology.orchestra.run.smithi164.stdout:(1/34): ceph-17.2.7-1623.gd0e6828a.el9.x86_64.r 55 kB/s | 6.5 kB 00:00 2024-09-11T07:23:01.116 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-11T07:23:01.148 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-11T07:23:01.181 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-11T07:23:01.270 INFO:teuthology.orchestra.run.smithi164.stdout:(2/34): ceph-mds-17.2.7-1623.gd0e6828a.el9.x86_ 6.3 MB/s | 2.1 MB 00:00 2024-09-11T07:23:01.281 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-11T07:23:01.321 INFO:teuthology.orchestra.run.smithi164.stdout:(3/34): ceph-mgr-17.2.7-1623.gd0e6828a.el9.x86_ 3.7 MB/s | 1.4 MB 00:00 2024-09-11T07:23:01.384 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-11T07:23:01.415 INFO:teuthology.orchestra.run.smithi164.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1623.gd0e6 2.5 MB/s | 240 kB 00:00 2024-09-11T07:23:01.425 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-11T07:23:01.460 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-11T07:23:01.488 INFO:teuthology.orchestra.run.smithi164.stdout:(5/34): ceph-mon-17.2.7-1623.gd0e6828a.el9.x86_ 9.2 MB/s | 3.9 MB 00:00 2024-09-11T07:23:01.490 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-11T07:23:01.524 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-11T07:23:01.558 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-11T07:23:01.590 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-11T07:23:01.622 INFO:teuthology.orchestra.run.smithi164.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-11T07:23:01.633 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-11T07:23:01.668 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-11T07:23:01.716 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-11T07:23:01.750 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T07:23:01.781 INFO:teuthology.orchestra.run.smithi164.stdout:(7/34): ceph-osd-17.2.7-1623.gd0e6828a.el9.x86_ 30 MB/s | 15 MB 00:00 2024-09-11T07:23:01.806 INFO:teuthology.orchestra.run.smithi164.stdout:(8/34): python3-ply-3.11-14.el9.noarch.rpm 580 kB/s | 106 kB 00:00 2024-09-11T07:23:01.823 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-11T07:23:01.839 INFO:teuthology.orchestra.run.smithi164.stdout:(9/34): python3-cryptography-36.0.1-4.el9.x86_6 3.5 MB/s | 1.2 MB 00:00 2024-09-11T07:23:01.881 INFO:teuthology.orchestra.run.smithi164.stdout:(10/34): python3-requests-2.25.1-8.el9.noarch.r 1.6 MB/s | 125 kB 00:00 2024-09-11T07:23:01.923 INFO:teuthology.orchestra.run.smithi164.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-11T07:23:01.949 INFO:teuthology.orchestra.run.smithi164.stdout:(12/34): python3-pycparser-2.20-6.el9.noarch.rp 806 kB/s | 135 kB 00:00 2024-09-11T07:23:01.974 INFO:teuthology.orchestra.run.smithi164.stdout:(13/34): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-09-11T07:23:01.978 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-11T07:23:01.999 INFO:teuthology.orchestra.run.smithi164.stdout:(14/34): python3-markupsafe-1.1.1-12.el9.x86_64 460 kB/s | 35 kB 00:00 2024-09-11T07:23:02.033 INFO:teuthology.orchestra.run.smithi164.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 920 kB/s | 77 kB 00:00 2024-09-11T07:23:02.058 INFO:teuthology.orchestra.run.smithi164.stdout:(16/34): python3-pytz-2021.1-5.el9.noarch.rpm 608 kB/s | 51 kB 00:00 2024-09-11T07:23:02.059 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-11T07:23:02.083 INFO:teuthology.orchestra.run.smithi164.stdout:(17/34): python3-toml-0.10.2-6.el9.noarch.rpm 496 kB/s | 42 kB 00:00 2024-09-11T07:23:02.128 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-11T07:23:02.134 INFO:teuthology.orchestra.run.smithi164.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 433 kB/s | 43 kB 00:00 2024-09-11T07:23:02.159 INFO:teuthology.orchestra.run.smithi164.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 462 kB/s | 11 kB 00:00 2024-09-11T07:23:02.167 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-11T07:23:02.192 INFO:teuthology.orchestra.run.smithi164.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-11T07:23:02.216 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-11T07:23:02.217 INFO:teuthology.orchestra.run.smithi164.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 302 kB/s | 18 kB 00:00 2024-09-11T07:23:02.243 INFO:teuthology.orchestra.run.smithi164.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 932 kB/s | 23 kB 00:00 2024-09-11T07:23:02.268 INFO:teuthology.orchestra.run.smithi164.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-11T07:23:02.270 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-11T07:23:02.293 INFO:teuthology.orchestra.run.smithi164.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-11T07:23:02.318 INFO:teuthology.orchestra.run.smithi164.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-09-11T07:23:02.344 INFO:teuthology.orchestra.run.smithi164.stdout:(26/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-09-11T07:23:02.369 INFO:teuthology.orchestra.run.smithi164.stdout:(27/34): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-11T07:23:02.395 INFO:teuthology.orchestra.run.smithi164.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-11T07:23:02.420 INFO:teuthology.orchestra.run.smithi164.stdout:(29/34): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-11T07:23:02.445 INFO:teuthology.orchestra.run.smithi164.stdout:(30/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-11T07:23:02.456 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-11T07:23:02.470 INFO:teuthology.orchestra.run.smithi164.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-11T07:23:02.496 INFO:teuthology.orchestra.run.smithi164.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 796 kB/s | 20 kB 00:00 2024-09-11T07:23:02.504 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-11T07:23:02.538 INFO:teuthology.orchestra.run.smithi164.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-11T07:23:02.563 INFO:teuthology.orchestra.run.smithi164.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-09-11T07:23:02.568 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:02.568 INFO:teuthology.orchestra.run.smithi164.stdout:Total 14 MB/s | 27 MB 00:01 2024-09-11T07:23:02.574 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-11T07:23:02.662 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-11T07:23:02.697 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:23:02.712 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-11T07:23:02.734 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:23:02.734 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:23:02.826 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-11T07:23:02.976 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 29/34 2024-09-11T07:23:02.997 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T07:23:03.022 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T07:23:03.022 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:23:03.022 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T07:23:03.022 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:23:03.023 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:23:03.023 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:03.127 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:23:03.128 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:23:04.340 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T07:23:04.362 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:23:04.369 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T07:23:04.369 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:23:04.369 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T07:23:04.369 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:23:04.369 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:23:04.369 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:04.433 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-11T07:23:04.465 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-11T07:23:04.506 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-11T07:23:04.599 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-11T07:23:04.686 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-11T07:23:04.726 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-11T07:23:04.758 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T07:23:04.761 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-11T07:23:04.784 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T07:23:04.784 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:23:04.785 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T07:23:04.785 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:23:04.785 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:23:04.785 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:04.799 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-11T07:23:04.842 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-11T07:23:04.900 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-11T07:23:04.941 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-11T07:23:04.984 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-11T07:23:05.001 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T07:23:05.019 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-11T07:23:05.025 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T07:23:05.025 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:23:05.025 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T07:23:05.025 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:23:05.025 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:23:05.026 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:05.067 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-11T07:23:05.091 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T07:23:05.108 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T07:23:05.181 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-11T07:23:05.321 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-11T07:23:05.406 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-11T07:23:05.477 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-11T07:23:05.524 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-11T07:23:05.572 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-11T07:23:05.638 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-11T07:23:05.830 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-11T07:23:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-11T07:23:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-11T07:23:06.045 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-11T07:23:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-11T07:23:06.202 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-11T07:23:06.350 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 29/34 2024-09-11T07:23:06.382 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T07:23:06.407 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T07:23:06.408 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:23:06.408 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T07:23:06.408 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:23:06.408 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:23:06.408 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:06.902 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T07:23:06.902 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/34 2024-09-11T07:23:06.902 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/34 2024-09-11T07:23:06.902 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/34 2024-09-11T07:23:06.902 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/34 2024-09-11T07:23:06.902 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/34 2024-09-11T07:23:06.902 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 6/34 2024-09-11T07:23:06.902 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-11T07:23:06.902 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-11T07:23:06.902 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-11T07:23:06.902 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-11T07:23:06.903 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-11T07:23:06.903 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-11T07:23:06.903 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-11T07:23:06.903 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-11T07:23:06.903 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T07:23:06.903 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-11T07:23:06.903 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-11T07:23:06.903 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-11T07:23:06.903 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-11T07:23:06.903 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-11T07:23:06.903 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-11T07:23:06.903 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-11T07:23:06.903 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-11T07:23:06.904 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-11T07:23:06.904 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-11T07:23:06.905 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-11T07:23:06.905 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-11T07:23:06.905 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-11T07:23:06.905 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-11T07:23:06.905 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-11T07:23:06.905 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-11T07:23:06.905 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-11T07:23:06.905 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-11T07:23:07.791 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T07:23:07.827 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T07:23:07.828 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:23:07.828 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T07:23:07.828 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:23:07.828 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:23:07.828 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:07.979 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-11T07:23:07.979 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:07.979 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:23:07.979 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:07.979 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:07.979 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:07.979 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:07.980 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:07.980 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:07.980 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T07:23:07.980 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T07:23:07.980 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T07:23:07.980 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T07:23:07.980 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T07:23:07.980 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T07:23:07.980 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T07:23:07.980 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T07:23:07.981 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T07:23:07.981 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T07:23:07.981 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T07:23:07.981 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T07:23:07.981 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T07:23:07.981 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T07:23:07.981 INFO:teuthology.orchestra.run.smithi143.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T07:23:07.981 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T07:23:07.981 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T07:23:07.981 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T07:23:07.981 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T07:23:07.982 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T07:23:07.982 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T07:23:07.982 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T07:23:07.982 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T07:23:07.982 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T07:23:07.982 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T07:23:07.982 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T07:23:07.982 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T07:23:07.982 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T07:23:07.983 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:07.983 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:08.217 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T07:23:08.242 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T07:23:08.242 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:23:08.242 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T07:23:08.242 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:23:08.243 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:23:08.243 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:08.374 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-base 2024-09-11T07:23:08.443 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T07:23:08.467 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T07:23:08.467 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:23:08.467 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T07:23:08.468 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:23:08.468 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:23:08.468 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T07:23:08.937 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:00:49 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:09.027 INFO:teuthology.orchestra.run.smithi143.stdout:Package ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:09.069 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:09.070 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2024-09-11T07:23:09.070 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:09.138 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install cephadm 2024-09-11T07:23:09.699 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:00:50 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:09.830 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:09.831 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:09.831 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-09-11T07:23:09.831 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:09.831 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:23:09.831 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 83 k 2024-09-11T07:23:09.831 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:09.831 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:23:09.832 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:09.832 INFO:teuthology.orchestra.run.smithi143.stdout:Install 1 Package 2024-09-11T07:23:09.832 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:09.832 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 83 k 2024-09-11T07:23:09.832 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 375 k 2024-09-11T07:23:09.832 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:23:10.017 INFO:teuthology.orchestra.run.smithi143.stdout:cephadm-17.2.7-1623.gd0e6828a.el9.noarch.rpm 447 kB/s | 83 kB 00:00 2024-09-11T07:23:10.018 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:10.018 INFO:teuthology.orchestra.run.smithi143.stdout:Total 443 kB/s | 83 kB 00:00 2024-09-11T07:23:10.019 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:23:10.023 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:23:10.024 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:23:10.025 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:23:10.026 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:23:10.105 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:23:10.236 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T07:23:10.237 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/34 2024-09-11T07:23:10.237 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/34 2024-09-11T07:23:10.237 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/34 2024-09-11T07:23:10.237 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/34 2024-09-11T07:23:10.237 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/34 2024-09-11T07:23:10.237 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 6/34 2024-09-11T07:23:10.237 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-11T07:23:10.237 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-11T07:23:10.237 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-11T07:23:10.237 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-11T07:23:10.237 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-11T07:23:10.238 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-11T07:23:10.238 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-11T07:23:10.238 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-11T07:23:10.238 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T07:23:10.238 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-11T07:23:10.238 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-11T07:23:10.238 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-11T07:23:10.238 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-11T07:23:10.238 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-11T07:23:10.238 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-11T07:23:10.238 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-11T07:23:10.238 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-11T07:23:10.238 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-11T07:23:10.238 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-11T07:23:10.239 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-11T07:23:10.239 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-11T07:23:10.239 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-11T07:23:10.240 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-11T07:23:10.240 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-11T07:23:10.240 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-11T07:23:10.240 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-11T07:23:10.240 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-11T07:23:10.525 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:23:10.621 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:23:10.996 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:23:11.230 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-11T07:23:11.230 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:11.230 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:23:11.230 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:11.231 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:11.231 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:11.231 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:11.231 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:11.231 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:11.231 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T07:23:11.231 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T07:23:11.231 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T07:23:11.231 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T07:23:11.231 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T07:23:11.231 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T07:23:11.231 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T07:23:11.231 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T07:23:11.232 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T07:23:11.232 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T07:23:11.232 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T07:23:11.232 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T07:23:11.232 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T07:23:11.232 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T07:23:11.232 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T07:23:11.232 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T07:23:11.232 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T07:23:11.232 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T07:23:11.232 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T07:23:11.232 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T07:23:11.232 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T07:23:11.233 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T07:23:11.233 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T07:23:11.233 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T07:23:11.233 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T07:23:11.233 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T07:23:11.233 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T07:23:11.233 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T07:23:11.233 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:11.233 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:11.255 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:23:11.255 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:11.255 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:23:11.255 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:11.255 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:11.255 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:11.474 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T07:23:11.593 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-base 2024-09-11T07:23:12.043 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:00:53 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:12.169 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:50 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:12.192 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:12.192 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:12.192 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repo Size 2024-09-11T07:23:12.192 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:12.193 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:23:12.193 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 137 k 2024-09-11T07:23:12.193 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:12.193 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:23:12.194 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:12.194 INFO:teuthology.orchestra.run.smithi143.stdout:Install 1 Package 2024-09-11T07:23:12.194 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:12.194 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 137 k 2024-09-11T07:23:12.194 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 379 k 2024-09-11T07:23:12.194 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:23:12.251 INFO:teuthology.orchestra.run.smithi164.stdout:Package ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:12.293 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:12.294 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-11T07:23:12.294 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:12.346 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install cephadm 2024-09-11T07:23:12.401 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-immutable-object-cache-17.2.7-1623.gd0e682 628 kB/s | 137 kB 00:00 2024-09-11T07:23:12.401 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:12.401 INFO:teuthology.orchestra.run.smithi143.stdout:Total 623 kB/s | 137 kB 00:00 2024-09-11T07:23:12.402 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:23:12.409 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:23:12.410 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:23:12.466 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:23:12.467 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:23:12.623 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:23:12.693 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:23:12.719 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:23:12.719 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:23:12.719 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T07:23:12.719 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:12.909 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:50 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:13.044 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:13.045 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:13.045 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-11T07:23:13.045 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:13.045 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:23:13.045 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 83 k 2024-09-11T07:23:13.045 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:13.045 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:23:13.045 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:13.045 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-11T07:23:13.045 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:13.045 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 83 k 2024-09-11T07:23:13.045 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 375 k 2024-09-11T07:23:13.045 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:23:13.234 INFO:teuthology.orchestra.run.smithi164.stdout:cephadm-17.2.7-1623.gd0e6828a.el9.noarch.rpm 439 kB/s | 83 kB 00:00 2024-09-11T07:23:13.235 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:13.235 INFO:teuthology.orchestra.run.smithi164.stdout:Total 435 kB/s | 83 kB 00:00 2024-09-11T07:23:13.235 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:23:13.240 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:23:13.240 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:23:13.242 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:23:13.242 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:23:13.318 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:23:13.679 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:23:13.679 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:13.679 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:23:13.680 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:13.680 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:13.680 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:13.767 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:23:13.862 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:23:13.904 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-mgr 2024-09-11T07:23:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:23:14.433 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:23:14.433 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:14.433 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:23:14.433 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:14.433 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:14.433 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:14.482 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:00:55 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:14.575 INFO:teuthology.orchestra.run.smithi143.stdout:Package ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:14.617 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:14.618 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2024-09-11T07:23:14.618 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:14.648 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T07:23:14.696 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T07:23:15.224 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:53 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:15.275 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:00:56 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:15.358 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:15.359 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:15.359 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repo Size 2024-09-11T07:23:15.359 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:15.360 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:23:15.360 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 137 k 2024-09-11T07:23:15.360 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:15.360 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:23:15.360 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:15.360 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-11T07:23:15.360 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:15.360 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 137 k 2024-09-11T07:23:15.360 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 379 k 2024-09-11T07:23:15.360 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:23:15.409 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:15.410 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:15.410 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-09-11T07:23:15.410 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:15.410 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:23:15.410 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 3.4 M 2024-09-11T07:23:15.410 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-09-11T07:23:15.410 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 24 k 2024-09-11T07:23:15.410 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 15 k 2024-09-11T07:23:15.411 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-11T07:23:15.411 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T07:23:15.411 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T07:23:15.411 INFO:teuthology.orchestra.run.smithi143.stdout:Installing weak dependencies: 2024-09-11T07:23:15.411 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-11T07:23:15.411 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:15.411 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:23:15.411 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:15.411 INFO:teuthology.orchestra.run.smithi143.stdout:Install 7 Packages 2024-09-11T07:23:15.411 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:15.412 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 3.7 M 2024-09-11T07:23:15.412 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 80 M 2024-09-11T07:23:15.412 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:23:15.558 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-immutable-object-cache-17.2.7-1623.gd0e682 697 kB/s | 137 kB 00:00 2024-09-11T07:23:15.558 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:15.559 INFO:teuthology.orchestra.run.smithi164.stdout:Total 691 kB/s | 137 kB 00:00 2024-09-11T07:23:15.559 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:23:15.566 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:23:15.566 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:23:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:23:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:23:15.781 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:23:15.850 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:23:15.876 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:23:15.876 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:23:15.876 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T07:23:15.876 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:15.993 INFO:teuthology.orchestra.run.smithi143.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1623.gd0e6 82 kB/s | 15 kB 00:00 2024-09-11T07:23:16.022 INFO:teuthology.orchestra.run.smithi143.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1623.gd0e 118 kB/s | 24 kB 00:00 2024-09-11T07:23:16.064 INFO:teuthology.orchestra.run.smithi143.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 128 kB/s | 9.0 kB 00:00 2024-09-11T07:23:16.089 INFO:teuthology.orchestra.run.smithi143.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-11T07:23:16.114 INFO:teuthology.orchestra.run.smithi143.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 443 kB/s | 41 kB 00:00 2024-09-11T07:23:16.139 INFO:teuthology.orchestra.run.smithi143.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-11T07:23:16.232 INFO:teuthology.orchestra.run.smithi143.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1623.gd0e6828a 8.0 MB/s | 3.4 MB 00:00 2024-09-11T07:23:16.235 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:16.236 INFO:teuthology.orchestra.run.smithi143.stdout:Total 4.4 MB/s | 3.7 MB 00:00 2024-09-11T07:23:16.236 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:23:16.244 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:23:16.244 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:23:16.336 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:23:16.336 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:23:16.651 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:23:16.708 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-11T07:23:16.742 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-11T07:23:16.796 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-11T07:23:16.806 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:23:16.806 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:16.806 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:23:16.806 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:16.807 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:16.807 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:16.829 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-11T07:23:16.858 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 5/7 2024-09-11T07:23:17.036 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr 2024-09-11T07:23:17.617 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:55 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:17.710 INFO:teuthology.orchestra.run.smithi164.stdout:Package ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:17.752 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:17.753 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-11T07:23:17.753 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:17.826 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T07:23:17.845 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 6/7 2024-09-11T07:23:17.878 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T07:23:18.395 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:56 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:18.529 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:18.529 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:18.530 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-11T07:23:18.530 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:18.530 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:23:18.530 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 3.4 M 2024-09-11T07:23:18.530 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-11T07:23:18.532 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 24 k 2024-09-11T07:23:18.532 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 15 k 2024-09-11T07:23:18.532 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-11T07:23:18.532 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T07:23:18.532 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T07:23:18.532 INFO:teuthology.orchestra.run.smithi164.stdout:Installing weak dependencies: 2024-09-11T07:23:18.532 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-11T07:23:18.532 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:18.532 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:23:18.532 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:18.532 INFO:teuthology.orchestra.run.smithi164.stdout:Install 7 Packages 2024-09-11T07:23:18.533 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:18.533 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 3.7 M 2024-09-11T07:23:18.533 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 80 M 2024-09-11T07:23:18.533 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:23:18.728 INFO:teuthology.orchestra.run.smithi164.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1623.gd0e6 89 kB/s | 15 kB 00:00 2024-09-11T07:23:18.753 INFO:teuthology.orchestra.run.smithi164.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1623.gd0e 128 kB/s | 24 kB 00:00 2024-09-11T07:23:18.795 INFO:teuthology.orchestra.run.smithi164.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-11T07:23:18.821 INFO:teuthology.orchestra.run.smithi164.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-11T07:23:18.846 INFO:teuthology.orchestra.run.smithi164.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 444 kB/s | 41 kB 00:00 2024-09-11T07:23:18.871 INFO:teuthology.orchestra.run.smithi164.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-11T07:23:18.980 INFO:teuthology.orchestra.run.smithi164.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1623.gd0e6828a 8.1 MB/s | 3.4 MB 00:00 2024-09-11T07:23:18.983 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:18.983 INFO:teuthology.orchestra.run.smithi164.stdout:Total 8.1 MB/s | 3.7 MB 00:00 2024-09-11T07:23:18.984 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:23:18.992 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T07:23:18.992 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 1/7 2024-09-11T07:23:18.992 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 2/7 2024-09-11T07:23:18.992 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 3/7 2024-09-11T07:23:18.993 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-11T07:23:18.993 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-11T07:23:18.993 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-11T07:23:18.994 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:23:18.994 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:23:19.083 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:23:19.084 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:23:19.268 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-11T07:23:19.268 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:19.268 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:23:19.269 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:19.269 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:19.269 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:19.269 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-11T07:23:19.269 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-11T07:23:19.269 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T07:23:19.269 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T07:23:19.269 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:19.269 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:19.392 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:23:19.464 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-11T07:23:19.502 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T07:23:19.507 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-11T07:23:19.562 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-11T07:23:19.594 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-11T07:23:19.623 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 5/7 2024-09-11T07:23:20.066 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:01 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:20.201 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:20.202 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================= 2024-09-11T07:23:20.202 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-09-11T07:23:20.202 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================= 2024-09-11T07:23:20.202 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:23:20.202 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 7.4 M 2024-09-11T07:23:20.202 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-09-11T07:23:20.202 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T07:23:20.202 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T07:23:20.202 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T07:23:20.203 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T07:23:20.203 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T07:23:20.203 INFO:teuthology.orchestra.run.smithi143.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T07:23:20.203 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T07:23:20.203 INFO:teuthology.orchestra.run.smithi143.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T07:23:20.203 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T07:23:20.203 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T07:23:20.203 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T07:23:20.203 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:20.203 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:23:20.203 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================= 2024-09-11T07:23:20.203 INFO:teuthology.orchestra.run.smithi143.stdout:Install 12 Packages 2024-09-11T07:23:20.203 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:20.204 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 38 M 2024-09-11T07:23:20.204 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 212 M 2024-09-11T07:23:20.204 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:23:20.627 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 6/7 2024-09-11T07:23:20.668 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T07:23:20.716 INFO:teuthology.orchestra.run.smithi143.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-16 17 MB/s | 7.4 MB 00:00 2024-09-11T07:23:20.824 INFO:teuthology.orchestra.run.smithi143.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-09-11T07:23:21.058 INFO:teuthology.orchestra.run.smithi143.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2024-09-11T07:23:21.100 INFO:teuthology.orchestra.run.smithi143.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 438 kB/s | 18 kB 00:00 2024-09-11T07:23:21.133 INFO:teuthology.orchestra.run.smithi143.stdout:(5/12): libquadmath-11.5.0-2.el9.x86_64.rpm 222 kB/s | 191 kB 00:00 2024-09-11T07:23:21.159 INFO:teuthology.orchestra.run.smithi143.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 654 kB/s | 38 kB 00:00 2024-09-11T07:23:21.209 INFO:teuthology.orchestra.run.smithi143.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-11T07:23:21.268 INFO:teuthology.orchestra.run.smithi143.stdout:(8/12): libgfortran-11.5.0-2.el9.x86_64.rpm 805 kB/s | 801 kB 00:00 2024-09-11T07:23:21.343 INFO:teuthology.orchestra.run.smithi143.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-09-11T07:23:21.527 INFO:teuthology.orchestra.run.smithi143.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 13 MB/s | 4.9 MB 00:00 2024-09-11T07:23:21.552 INFO:teuthology.orchestra.run.smithi143.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 883 kB/s | 250 kB 00:00 2024-09-11T07:23:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T07:23:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 1/7 2024-09-11T07:23:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 2/7 2024-09-11T07:23:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 3/7 2024-09-11T07:23:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-11T07:23:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-11T07:23:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-11T07:23:21.727 INFO:teuthology.orchestra.run.smithi143.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-09-11T07:23:21.728 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:21.729 INFO:teuthology.orchestra.run.smithi143.stdout:Total 25 MB/s | 38 MB 00:01 2024-09-11T07:23:21.983 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-11T07:23:21.983 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:21.983 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:23:21.983 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:21.983 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:21.983 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:21.983 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-11T07:23:21.983 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-11T07:23:21.983 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T07:23:21.983 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T07:23:21.984 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:21.984 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:22.082 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:23:22.109 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:23:22.109 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:23:22.241 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T07:23:22.634 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:23:22.634 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:23:22.803 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:00 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:22.935 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:22.936 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-09-11T07:23:22.936 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-11T07:23:22.937 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-09-11T07:23:22.937 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:23:22.937 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 7.4 M 2024-09-11T07:23:22.937 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-11T07:23:22.937 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T07:23:22.937 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T07:23:22.937 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T07:23:22.937 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T07:23:22.937 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T07:23:22.937 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T07:23:22.937 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T07:23:22.937 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T07:23:22.937 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T07:23:22.938 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T07:23:22.938 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T07:23:22.938 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:22.938 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:23:22.938 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-09-11T07:23:22.938 INFO:teuthology.orchestra.run.smithi164.stdout:Install 12 Packages 2024-09-11T07:23:22.938 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:22.938 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 38 M 2024-09-11T07:23:22.939 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 212 M 2024-09-11T07:23:22.939 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:23:23.394 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:23:23.405 INFO:teuthology.orchestra.run.smithi164.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-11T07:23:23.438 INFO:teuthology.orchestra.run.smithi164.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 801 kB 00:00 2024-09-11T07:23:23.483 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T07:23:23.514 INFO:teuthology.orchestra.run.smithi164.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-09-11T07:23:23.525 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T07:23:23.572 INFO:teuthology.orchestra.run.smithi164.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 311 kB/s | 18 kB 00:00 2024-09-11T07:23:23.625 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T07:23:23.639 INFO:teuthology.orchestra.run.smithi164.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 576 kB/s | 38 kB 00:00 2024-09-11T07:23:23.660 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T07:23:23.740 INFO:teuthology.orchestra.run.smithi164.stdout:(6/12): ceph-mgr-diskprediction-local-17.2.7-16 16 MB/s | 7.4 MB 00:00 2024-09-11T07:23:23.807 INFO:teuthology.orchestra.run.smithi164.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 8.1 MB/s | 3.0 MB 00:00 2024-09-11T07:23:23.845 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T07:23:23.884 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T07:23:23.949 INFO:teuthology.orchestra.run.smithi164.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-09-11T07:23:23.985 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T07:23:24.024 INFO:teuthology.orchestra.run.smithi164.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 13 MB/s | 4.9 MB 00:00 2024-09-11T07:23:24.049 INFO:teuthology.orchestra.run.smithi164.stdout:(10/12): python3-devel-3.9.19-8.el9.x86_64.rpm 792 kB/s | 245 kB 00:00 2024-09-11T07:23:24.083 INFO:teuthology.orchestra.run.smithi164.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.8 MB/s | 250 kB 00:00 2024-09-11T07:23:24.341 INFO:teuthology.orchestra.run.smithi164.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 51 MB/s | 16 MB 00:00 2024-09-11T07:23:24.342 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:24.342 INFO:teuthology.orchestra.run.smithi164.stdout:Total 27 MB/s | 38 MB 00:01 2024-09-11T07:23:24.632 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T07:23:24.696 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:23:24.723 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:23:24.723 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:23:24.748 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T07:23:25.237 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:23:25.238 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:23:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:23:26.006 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T07:23:26.077 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T07:23:26.123 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T07:23:26.221 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T07:23:26.257 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T07:23:26.443 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T07:23:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T07:23:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T07:23:27.061 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T07:23:27.084 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T07:23:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T07:23:27.378 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T07:23:28.629 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T07:23:29.671 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T07:23:29.698 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T07:23:29.787 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T07:23:29.787 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 1/12 2024-09-11T07:23:29.787 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T07:23:29.787 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T07:23:29.787 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T07:23:29.787 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T07:23:29.787 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T07:23:29.788 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T07:23:29.788 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T07:23:29.788 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T07:23:29.788 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T07:23:29.788 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T07:23:30.194 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T07:23:30.194 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:30.194 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:23:30.194 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:30.194 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T07:23:30.194 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T07:23:30.195 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T07:23:30.195 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T07:23:30.195 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T07:23:30.195 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T07:23:30.195 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T07:23:30.195 INFO:teuthology.orchestra.run.smithi143.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T07:23:30.195 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T07:23:30.195 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T07:23:30.195 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T07:23:30.195 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:30.195 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:30.608 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-mgr-rook 2024-09-11T07:23:31.175 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:12 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:31.308 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:31.309 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:31.310 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-09-11T07:23:31.310 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:31.310 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:23:31.310 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 47 k 2024-09-11T07:23:31.310 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-09-11T07:23:31.310 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T07:23:31.310 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T07:23:31.310 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T07:23:31.310 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T07:23:31.310 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T07:23:31.310 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T07:23:31.310 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T07:23:31.311 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T07:23:31.311 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T07:23:31.311 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T07:23:31.311 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T07:23:31.311 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T07:23:31.311 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:31.311 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:23:31.311 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:31.311 INFO:teuthology.orchestra.run.smithi143.stdout:Install 13 Packages 2024-09-11T07:23:31.311 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:31.312 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 2.2 M 2024-09-11T07:23:31.312 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 26 M 2024-09-11T07:23:31.312 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:23:31.703 INFO:teuthology.orchestra.run.smithi143.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 210 kB/s | 19 kB 00:00 2024-09-11T07:23:31.728 INFO:teuthology.orchestra.run.smithi143.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 220 kB/s | 26 kB 00:00 2024-09-11T07:23:31.787 INFO:teuthology.orchestra.run.smithi143.stdout:(3/13): ceph-mgr-rook-17.2.7-1623.gd0e6828a.el9 271 kB/s | 47 kB 00:00 2024-09-11T07:23:31.829 INFO:teuthology.orchestra.run.smithi143.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.7 MB/s | 222 kB 00:00 2024-09-11T07:23:31.854 INFO:teuthology.orchestra.run.smithi143.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-11T07:23:31.879 INFO:teuthology.orchestra.run.smithi143.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-11T07:23:31.930 INFO:teuthology.orchestra.run.smithi143.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 428 kB/s | 32 kB 00:00 2024-09-11T07:23:31.955 INFO:teuthology.orchestra.run.smithi143.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 187 kB/s | 14 kB 00:00 2024-09-11T07:23:31.988 INFO:teuthology.orchestra.run.smithi143.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-09-11T07:23:32.013 INFO:teuthology.orchestra.run.smithi143.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-11T07:23:32.048 INFO:teuthology.orchestra.run.smithi143.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.3 MB/s | 90 kB 00:00 2024-09-11T07:23:32.081 INFO:teuthology.orchestra.run.smithi143.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 641 kB/s | 59 kB 00:00 2024-09-11T07:23:32.123 INFO:teuthology.orchestra.run.smithi143.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.1 MB/s | 1.0 MB 00:00 2024-09-11T07:23:32.126 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:32.126 INFO:teuthology.orchestra.run.smithi143.stdout:Total 2.7 MB/s | 2.2 MB 00:00 2024-09-11T07:23:32.195 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:23:32.211 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:23:32.211 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:23:32.297 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:23:32.299 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:23:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T07:23:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 1/12 2024-09-11T07:23:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T07:23:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T07:23:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T07:23:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T07:23:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T07:23:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T07:23:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T07:23:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T07:23:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T07:23:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T07:23:32.758 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T07:23:32.759 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:32.759 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:23:32.759 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:32.759 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T07:23:32.759 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T07:23:32.759 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T07:23:32.759 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T07:23:32.759 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T07:23:32.759 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T07:23:32.759 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T07:23:32.759 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T07:23:32.759 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T07:23:32.760 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T07:23:32.760 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T07:23:32.760 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:32.760 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:32.801 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:23:32.985 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T07:23:33.040 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T07:23:33.104 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T07:23:33.142 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T07:23:33.191 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T07:23:33.200 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr-rook 2024-09-11T07:23:33.300 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T07:23:33.411 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T07:23:33.460 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T07:23:33.771 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:11 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:33.906 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:33.907 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:33.907 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-11T07:23:33.907 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:33.907 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:23:33.907 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 47 k 2024-09-11T07:23:33.907 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-11T07:23:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T07:23:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T07:23:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T07:23:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T07:23:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T07:23:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T07:23:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T07:23:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T07:23:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T07:23:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T07:23:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T07:23:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T07:23:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:33.909 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:23:33.909 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:33.909 INFO:teuthology.orchestra.run.smithi164.stdout:Install 13 Packages 2024-09-11T07:23:33.909 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:33.909 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 2.2 M 2024-09-11T07:23:33.910 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 26 M 2024-09-11T07:23:33.910 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:23:34.167 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T07:23:34.216 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T07:23:34.250 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T07:23:34.295 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T07:23:34.320 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T07:23:34.483 INFO:teuthology.orchestra.run.smithi164.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 234 kB/s | 19 kB 00:00 2024-09-11T07:23:34.499 INFO:teuthology.orchestra.run.smithi164.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 260 kB/s | 26 kB 00:00 2024-09-11T07:23:34.558 INFO:teuthology.orchestra.run.smithi164.stdout:(3/13): ceph-mgr-rook-17.2.7-1623.gd0e6828a.el9 302 kB/s | 47 kB 00:00 2024-09-11T07:23:34.584 INFO:teuthology.orchestra.run.smithi164.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 222 kB 00:00 2024-09-11T07:23:34.609 INFO:teuthology.orchestra.run.smithi164.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-11T07:23:34.625 INFO:teuthology.orchestra.run.smithi164.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.2 MB/s | 54 kB 00:00 2024-09-11T07:23:34.659 INFO:teuthology.orchestra.run.smithi164.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 639 kB/s | 32 kB 00:00 2024-09-11T07:23:34.676 INFO:teuthology.orchestra.run.smithi164.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 280 kB/s | 14 kB 00:00 2024-09-11T07:23:34.701 INFO:teuthology.orchestra.run.smithi164.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.9 MB/s | 279 kB 00:00 2024-09-11T07:23:34.727 INFO:teuthology.orchestra.run.smithi164.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 3.3 MB/s | 220 kB 00:00 2024-09-11T07:23:34.760 INFO:teuthology.orchestra.run.smithi164.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.3 MB/s | 90 kB 00:00 2024-09-11T07:23:34.777 INFO:teuthology.orchestra.run.smithi164.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 783 kB/s | 59 kB 00:00 2024-09-11T07:23:34.811 INFO:teuthology.orchestra.run.smithi164.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.6 MB/s | 1.0 MB 00:00 2024-09-11T07:23:34.814 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:34.814 INFO:teuthology.orchestra.run.smithi164.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-09-11T07:23:34.877 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:23:34.892 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:23:34.892 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:23:34.980 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:23:34.981 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:23:35.193 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T07:23:35.193 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 1/13 2024-09-11T07:23:35.193 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T07:23:35.193 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T07:23:35.194 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T07:23:35.194 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T07:23:35.194 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T07:23:35.194 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T07:23:35.194 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T07:23:35.194 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T07:23:35.194 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T07:23:35.194 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T07:23:35.194 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T07:23:35.541 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T07:23:35.542 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:35.542 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:23:35.542 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:35.542 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T07:23:35.542 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T07:23:35.542 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T07:23:35.542 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T07:23:35.542 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T07:23:35.542 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T07:23:35.542 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T07:23:35.542 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T07:23:35.543 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T07:23:35.543 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T07:23:35.543 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T07:23:35.543 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T07:23:35.543 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:35.543 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:35.558 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:23:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T07:23:35.796 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T07:23:35.831 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T07:23:35.859 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T07:23:35.897 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T07:23:35.938 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T07:23:36.040 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T07:23:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T07:23:36.207 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T07:23:36.399 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:17 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:36.532 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:36.533 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:36.533 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-09-11T07:23:36.533 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:36.533 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:23:36.533 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 124 k 2024-09-11T07:23:36.533 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-09-11T07:23:36.533 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T07:23:36.533 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T07:23:36.533 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T07:23:36.533 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T07:23:36.534 INFO:teuthology.orchestra.run.smithi143.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T07:23:36.534 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:36.534 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:23:36.534 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:36.534 INFO:teuthology.orchestra.run.smithi143.stdout:Install 6 Packages 2024-09-11T07:23:36.534 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:36.535 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 7.0 M 2024-09-11T07:23:36.535 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 33 M 2024-09-11T07:23:36.535 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:23:36.937 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T07:23:36.979 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T07:23:37.013 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T07:23:37.059 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T07:23:37.082 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T07:23:37.185 INFO:teuthology.orchestra.run.smithi143.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-09-11T07:23:37.210 INFO:teuthology.orchestra.run.smithi143.stdout:(2/6): ceph-mgr-cephadm-17.2.7-1623.gd0e6828a.e 511 kB/s | 124 kB 00:00 2024-09-11T07:23:37.261 INFO:teuthology.orchestra.run.smithi143.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-11T07:23:37.278 INFO:teuthology.orchestra.run.smithi143.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 862 kB/s | 58 kB 00:00 2024-09-11T07:23:37.303 INFO:teuthology.orchestra.run.smithi143.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.7 MB/s | 74 kB 00:00 2024-09-11T07:23:37.536 INFO:teuthology.orchestra.run.smithi143.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-09-11T07:23:37.539 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:37.539 INFO:teuthology.orchestra.run.smithi143.stdout:Total 7.0 MB/s | 7.0 MB 00:01 2024-09-11T07:23:37.610 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:23:37.622 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:23:37.622 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:23:37.722 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:23:37.723 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:23:37.854 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T07:23:37.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 1/13 2024-09-11T07:23:37.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T07:23:37.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T07:23:37.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T07:23:37.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T07:23:37.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T07:23:37.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T07:23:37.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T07:23:37.854 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T07:23:37.855 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T07:23:37.855 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T07:23:37.855 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T07:23:38.039 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:23:38.199 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T07:23:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:38.200 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:23:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T07:23:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T07:23:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T07:23:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T07:23:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T07:23:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T07:23:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T07:23:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T07:23:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T07:23:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T07:23:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T07:23:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T07:23:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:38.201 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:38.218 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T07:23:38.280 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T07:23:38.441 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T07:23:38.799 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T07:23:38.891 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T07:23:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T07:23:38.984 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T07:23:39.036 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:17 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:39.171 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:39.172 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:39.172 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-11T07:23:39.172 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:39.172 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:23:39.172 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 124 k 2024-09-11T07:23:39.172 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-11T07:23:39.172 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T07:23:39.172 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T07:23:39.173 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T07:23:39.173 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T07:23:39.173 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T07:23:39.173 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:39.173 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:23:39.173 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:39.173 INFO:teuthology.orchestra.run.smithi164.stdout:Install 6 Packages 2024-09-11T07:23:39.173 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:39.174 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 7.0 M 2024-09-11T07:23:39.174 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 33 M 2024-09-11T07:23:39.174 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:23:39.448 INFO:teuthology.orchestra.run.smithi164.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 2.0 MB/s | 248 kB 00:00 2024-09-11T07:23:39.524 INFO:teuthology.orchestra.run.smithi164.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.2 MB/s | 548 kB 00:00 2024-09-11T07:23:39.549 INFO:teuthology.orchestra.run.smithi164.stdout:(3/6): ceph-mgr-cephadm-17.2.7-1623.gd0e6828a.e 556 kB/s | 124 kB 00:00 2024-09-11T07:23:39.574 INFO:teuthology.orchestra.run.smithi164.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-11T07:23:39.616 INFO:teuthology.orchestra.run.smithi164.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-11T07:23:39.699 INFO:teuthology.orchestra.run.smithi164.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-11T07:23:39.702 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:39.702 INFO:teuthology.orchestra.run.smithi164.stdout:Total 13 MB/s | 7.0 MB 00:00 2024-09-11T07:23:39.707 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T07:23:39.707 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/6 2024-09-11T07:23:39.707 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T07:23:39.707 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T07:23:39.707 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T07:23:39.707 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T07:23:39.772 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:23:39.783 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:23:39.783 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:23:39.882 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:23:39.883 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:23:39.981 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T07:23:39.982 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:39.982 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:23:39.982 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:39.982 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T07:23:39.982 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T07:23:39.982 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T07:23:39.982 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T07:23:39.982 INFO:teuthology.orchestra.run.smithi143.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T07:23:39.982 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:39.982 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:40.186 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:23:40.236 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-fuse 2024-09-11T07:23:40.356 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T07:23:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T07:23:40.798 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:21 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T07:23:40.930 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:40.930 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:40.930 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-09-11T07:23:40.930 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:40.931 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:23:40.931 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 813 k 2024-09-11T07:23:40.931 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-09-11T07:23:40.931 INFO:teuthology.orchestra.run.smithi143.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T07:23:40.931 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:40.931 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:23:40.931 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:40.931 INFO:teuthology.orchestra.run.smithi143.stdout:Install 2 Packages 2024-09-11T07:23:40.931 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:40.932 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 893 k 2024-09-11T07:23:40.932 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 2.6 M 2024-09-11T07:23:40.932 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:23:41.029 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T07:23:41.084 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T07:23:41.106 INFO:teuthology.orchestra.run.smithi143.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 523 kB/s | 80 kB 00:00 2024-09-11T07:23:41.114 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T07:23:41.223 INFO:teuthology.orchestra.run.smithi143.stdout:(2/2): ceph-fuse-17.2.7-1623.gd0e6828a.el9.x86_ 2.9 MB/s | 813 kB 00:00 2024-09-11T07:23:41.223 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:41.223 INFO:teuthology.orchestra.run.smithi143.stdout:Total 3.0 MB/s | 893 kB 00:00 2024-09-11T07:23:41.235 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:23:41.257 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:23:41.257 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:23:41.343 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:23:41.343 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:23:41.535 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:23:41.686 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T07:23:41.754 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T07:23:41.948 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T07:23:41.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/6 2024-09-11T07:23:41.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T07:23:41.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T07:23:41.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T07:23:41.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T07:23:42.222 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T07:23:42.223 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:42.223 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:23:42.223 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:42.223 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T07:23:42.223 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T07:23:42.223 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T07:23:42.223 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T07:23:42.223 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T07:23:42.223 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:42.224 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:42.451 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-fuse 2024-09-11T07:23:42.452 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T07:23:42.452 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/2 2024-09-11T07:23:42.724 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T07:23:42.725 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:42.725 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:23:42.725 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T07:23:42.725 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:42.725 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:43.006 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-volume 2024-09-11T07:23:43.019 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:21 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:43.151 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:43.152 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:43.152 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-11T07:23:43.152 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:43.152 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:23:43.152 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 813 k 2024-09-11T07:23:43.152 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-11T07:23:43.152 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T07:23:43.152 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:43.152 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:23:43.153 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:43.153 INFO:teuthology.orchestra.run.smithi164.stdout:Install 2 Packages 2024-09-11T07:23:43.153 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:43.153 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 893 k 2024-09-11T07:23:43.153 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 2.6 M 2024-09-11T07:23:43.153 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:23:43.410 INFO:teuthology.orchestra.run.smithi164.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 567 kB/s | 80 kB 00:00 2024-09-11T07:23:43.560 INFO:teuthology.orchestra.run.smithi164.stdout:(2/2): ceph-fuse-17.2.7-1623.gd0e6828a.el9.x86_ 2.7 MB/s | 813 kB 00:00 2024-09-11T07:23:43.561 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:43.561 INFO:teuthology.orchestra.run.smithi164.stdout:Total 2.1 MB/s | 893 kB 00:00 2024-09-11T07:23:43.573 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:23:43.582 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:24 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:43.597 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:23:43.597 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:23:43.681 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:23:43.681 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:23:43.718 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:43.718 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:43.719 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-09-11T07:23:43.719 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:43.719 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:23:43.719 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 270 k 2024-09-11T07:23:43.719 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:43.719 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:23:43.719 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:43.719 INFO:teuthology.orchestra.run.smithi143.stdout:Install 1 Package 2024-09-11T07:23:43.719 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:43.719 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 270 k 2024-09-11T07:23:43.719 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 1.3 M 2024-09-11T07:23:43.719 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:23:43.867 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:23:43.946 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-volume-17.2.7-1623.gd0e6828a.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-11T07:23:43.947 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:43.947 INFO:teuthology.orchestra.run.smithi143.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-11T07:23:43.947 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:23:43.956 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:23:43.956 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:23:43.981 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:23:43.982 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:23:44.019 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T07:23:44.092 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T07:23:44.193 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:23:44.265 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:23:44.281 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:23:44.281 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:23:44.281 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T07:23:44.281 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:44.833 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T07:23:44.833 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/2 2024-09-11T07:23:45.109 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T07:23:45.110 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:45.110 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:23:45.110 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T07:23:45.110 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:45.110 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:45.316 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:23:45.316 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:45.316 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:23:45.316 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:45.316 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:45.316 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:45.384 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-volume 2024-09-11T07:23:45.611 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install librados-devel 2024-09-11T07:23:45.962 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:23 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:46.095 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:46.096 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:46.096 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-11T07:23:46.096 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:46.096 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:23:46.096 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 270 k 2024-09-11T07:23:46.096 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:46.096 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:23:46.096 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:46.096 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-11T07:23:46.096 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:46.097 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 270 k 2024-09-11T07:23:46.097 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 1.3 M 2024-09-11T07:23:46.097 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:23:46.195 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:27 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:46.325 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-volume-17.2.7-1623.gd0e6828a.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-11T07:23:46.325 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:46.325 INFO:teuthology.orchestra.run.smithi164.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-11T07:23:46.326 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:23:46.331 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:46.331 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:46.331 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repo Size 2024-09-11T07:23:46.332 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:46.332 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:23:46.332 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 126 k 2024-09-11T07:23:46.332 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:46.332 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:23:46.332 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:46.332 INFO:teuthology.orchestra.run.smithi143.stdout:Install 1 Package 2024-09-11T07:23:46.332 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:46.332 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 126 k 2024-09-11T07:23:46.333 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 448 k 2024-09-11T07:23:46.333 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:23:46.334 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:23:46.334 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:23:46.366 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:23:46.366 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:23:46.544 INFO:teuthology.orchestra.run.smithi143.stdout:librados-devel-17.2.7-1623.gd0e6828a.el9.x86_64 596 kB/s | 126 kB 00:00 2024-09-11T07:23:46.545 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:46.545 INFO:teuthology.orchestra.run.smithi143.stdout:Total 593 kB/s | 126 kB 00:00 2024-09-11T07:23:46.545 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:23:46.550 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:23:46.550 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:23:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:23:46.614 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:23:46.614 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:23:46.660 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:23:46.676 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:23:46.676 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:23:46.676 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T07:23:46.676 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:46.765 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:23:46.858 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:47.277 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:47.541 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:47.541 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:47.541 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:23:47.541 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:47.541 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:47.541 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:47.691 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:23:47.691 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:47.691 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:23:47.692 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:23:47.692 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:47.692 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:47.828 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install libcephfs2 2024-09-11T07:23:47.976 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install librados-devel 2024-09-11T07:23:48.408 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:29 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:48.501 INFO:teuthology.orchestra.run.smithi143.stdout:Package libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:48.546 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:48.547 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2024-09-11T07:23:48.547 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:48.547 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:26 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:48.621 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install libcephfs-devel 2024-09-11T07:23:48.687 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:48.687 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:48.688 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repo Size 2024-09-11T07:23:48.688 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:48.688 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:23:48.688 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 126 k 2024-09-11T07:23:48.688 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:48.688 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:23:48.688 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:48.688 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-11T07:23:48.688 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:48.688 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 126 k 2024-09-11T07:23:48.688 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 448 k 2024-09-11T07:23:48.688 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:23:48.881 INFO:teuthology.orchestra.run.smithi164.stdout:librados-devel-17.2.7-1623.gd0e6828a.el9.x86_64 656 kB/s | 126 kB 00:00 2024-09-11T07:23:48.882 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:48.882 INFO:teuthology.orchestra.run.smithi164.stdout:Total 650 kB/s | 126 kB 00:00 2024-09-11T07:23:48.882 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:23:48.888 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:23:48.888 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:23:48.951 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:23:48.951 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:23:49.110 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:23:49.187 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:30 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:49.203 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:49.320 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:49.321 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:49.321 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repo Size 2024-09-11T07:23:49.321 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:49.322 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:23:49.322 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:23:49.322 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:49.322 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:23:49.322 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:49.322 INFO:teuthology.orchestra.run.smithi143.stdout:Install 1 Package 2024-09-11T07:23:49.322 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:49.322 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 25 k 2024-09-11T07:23:49.322 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 110 k 2024-09-11T07:23:49.322 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:23:49.474 INFO:teuthology.orchestra.run.smithi143.stdout:libcephfs-devel-17.2.7-1623.gd0e6828a.el9.x86_6 164 kB/s | 25 kB 00:00 2024-09-11T07:23:49.475 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:49.475 INFO:teuthology.orchestra.run.smithi143.stdout:Total 162 kB/s | 25 kB 00:00 2024-09-11T07:23:49.475 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:23:49.479 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:23:49.479 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:23:49.480 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:23:49.481 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:23:49.567 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:23:49.619 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:49.679 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:49.883 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:23:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:49.884 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:50.057 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:50.116 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install libcephfs2 2024-09-11T07:23:50.296 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:50.297 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:50.297 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:23:50.297 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:50.297 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:50.297 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:50.482 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install librados2 2024-09-11T07:23:50.694 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:28 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:50.786 INFO:teuthology.orchestra.run.smithi164.stdout:Package libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:50.830 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:50.831 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-11T07:23:50.831 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:50.944 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install libcephfs-devel 2024-09-11T07:23:51.046 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:32 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:51.136 INFO:teuthology.orchestra.run.smithi143.stdout:Package librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:51.179 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:51.180 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2024-09-11T07:23:51.180 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:51.256 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install librbd1 2024-09-11T07:23:51.517 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:29 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:51.652 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:51.652 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:51.652 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repo Size 2024-09-11T07:23:51.652 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:51.653 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:23:51.653 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:23:51.653 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:51.653 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:23:51.653 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:51.653 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-11T07:23:51.653 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:51.653 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 25 k 2024-09-11T07:23:51.653 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 110 k 2024-09-11T07:23:51.653 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:23:51.802 INFO:teuthology.orchestra.run.smithi164.stdout:libcephfs-devel-17.2.7-1623.gd0e6828a.el9.x86_6 167 kB/s | 25 kB 00:00 2024-09-11T07:23:51.803 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:51.803 INFO:teuthology.orchestra.run.smithi164.stdout:Total 166 kB/s | 25 kB 00:00 2024-09-11T07:23:51.803 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:23:51.807 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:23:51.807 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:23:51.808 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:23:51.808 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:23:51.813 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:32 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:51.883 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:23:51.902 INFO:teuthology.orchestra.run.smithi143.stdout:Package librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:51.945 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:51.946 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2024-09-11T07:23:51.946 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:52.000 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:52.027 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install python3-rados 2024-09-11T07:23:52.401 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:52.584 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:33 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:52.660 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:52.660 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:52.660 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:23:52.660 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:52.661 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:52.661 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:52.672 INFO:teuthology.orchestra.run.smithi143.stdout:Package python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:52.716 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:52.717 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2024-09-11T07:23:52.717 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:52.787 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install python3-rgw 2024-09-11T07:23:52.852 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install librados2 2024-09-11T07:23:53.349 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:34 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:53.419 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:31 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:53.438 INFO:teuthology.orchestra.run.smithi143.stdout:Package python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:53.481 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:53.482 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2024-09-11T07:23:53.482 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:53.510 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:53.554 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:53.555 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install python3-cephfs 2024-09-11T07:23:53.555 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-11T07:23:53.555 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:53.667 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install librbd1 2024-09-11T07:23:54.124 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:35 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:54.215 INFO:teuthology.orchestra.run.smithi143.stdout:Package python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:54.236 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:32 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:54.259 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:54.260 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2024-09-11T07:23:54.260 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:54.327 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:54.339 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install python3-rbd 2024-09-11T07:23:54.371 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:54.372 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-11T07:23:54.372 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:54.436 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install python3-rados 2024-09-11T07:23:54.893 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:35 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:54.983 INFO:teuthology.orchestra.run.smithi143.stdout:Package python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:54.999 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:32 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:55.025 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:55.026 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2024-09-11T07:23:55.026 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:55.089 INFO:teuthology.orchestra.run.smithi164.stdout:Package python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:55.092 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install rbd-fuse 2024-09-11T07:23:55.130 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:55.131 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-11T07:23:55.131 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:55.190 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install python3-rgw 2024-09-11T07:23:55.653 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:55.753 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:33 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:55.785 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:55.786 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:55.786 INFO:teuthology.orchestra.run.smithi143.stdout: Package Architecture Version Repository Size 2024-09-11T07:23:55.786 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:55.786 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:23:55.787 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 82 k 2024-09-11T07:23:55.787 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:55.787 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:23:55.787 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:55.787 INFO:teuthology.orchestra.run.smithi143.stdout:Install 1 Package 2024-09-11T07:23:55.787 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:55.787 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 82 k 2024-09-11T07:23:55.787 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 222 k 2024-09-11T07:23:55.787 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:23:55.843 INFO:teuthology.orchestra.run.smithi164.stdout:Package python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:55.886 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:55.887 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-11T07:23:55.887 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:55.953 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install python3-cephfs 2024-09-11T07:23:55.978 INFO:teuthology.orchestra.run.smithi143.stdout:rbd-fuse-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 429 kB/s | 82 kB 00:00 2024-09-11T07:23:55.979 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:55.979 INFO:teuthology.orchestra.run.smithi143.stdout:Total 426 kB/s | 82 kB 00:00 2024-09-11T07:23:55.979 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:23:55.985 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:23:55.985 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:23:56.048 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:23:56.048 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:23:56.189 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:23:56.291 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:56.521 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:34 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:56.614 INFO:teuthology.orchestra.run.smithi164.stdout:Package python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:56.658 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:56.659 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-11T07:23:56.659 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:56.697 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:56.724 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install python3-rbd 2024-09-11T07:23:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:56.955 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:23:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:56.955 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:23:57.163 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install rbd-mirror 2024-09-11T07:23:57.290 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:35 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:57.381 INFO:teuthology.orchestra.run.smithi164.stdout:Package python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:23:57.423 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:57.425 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-11T07:23:57.425 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:57.490 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install rbd-fuse 2024-09-11T07:23:57.737 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:23:57.871 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:23:57.872 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:57.872 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repo Size 2024-09-11T07:23:57.872 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:57.872 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:23:57.872 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:23:57.872 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:57.872 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:23:57.872 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:23:57.872 INFO:teuthology.orchestra.run.smithi143.stdout:Install 1 Package 2024-09-11T07:23:57.873 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:57.873 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 2.9 M 2024-09-11T07:23:57.873 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 12 M 2024-09-11T07:23:57.873 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:23:58.058 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:23:58.192 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:23:58.193 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:58.193 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-09-11T07:23:58.193 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:58.193 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:23:58.193 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 82 k 2024-09-11T07:23:58.193 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:58.193 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:23:58.193 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:23:58.193 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-11T07:23:58.194 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:58.194 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 82 k 2024-09-11T07:23:58.194 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 222 k 2024-09-11T07:23:58.194 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:23:58.198 INFO:teuthology.orchestra.run.smithi143.stdout:rbd-mirror-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 9.0 MB/s | 2.9 MB 00:00 2024-09-11T07:23:58.199 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:58.200 INFO:teuthology.orchestra.run.smithi143.stdout:Total 9.0 MB/s | 2.9 MB 00:00 2024-09-11T07:23:58.200 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:23:58.208 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:23:58.209 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:23:58.289 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:23:58.289 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:23:58.389 INFO:teuthology.orchestra.run.smithi164.stdout:rbd-fuse-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 418 kB/s | 82 kB 00:00 2024-09-11T07:23:58.390 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:23:58.390 INFO:teuthology.orchestra.run.smithi164.stdout:Total 416 kB/s | 82 kB 00:00 2024-09-11T07:23:58.390 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:23:58.397 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:23:58.397 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:23:58.460 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:23:58.461 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:23:58.605 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:23:58.700 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:23:58.711 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:58.774 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:58.800 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:58.800 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:23:58.800 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T07:23:58.800 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T07:23:58.800 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T07:23:58.801 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:59.069 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:59.348 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:59.348 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:59.348 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:23:59.348 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:59.348 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:23:59.348 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:23:59.573 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install rbd-mirror 2024-09-11T07:23:59.871 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:23:59.872 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:59.872 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:23:59.872 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:23:59.872 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:23:59.872 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:24:00.096 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install rbd-nbd 2024-09-11T07:24:00.154 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:24:00.290 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:24:00.291 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:24:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repo Size 2024-09-11T07:24:00.291 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:24:00.291 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:24:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:24:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:24:00.291 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:24:00.291 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:24:00.291 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-11T07:24:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:24:00.291 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 2.9 M 2024-09-11T07:24:00.291 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 12 M 2024-09-11T07:24:00.292 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:24:00.634 INFO:teuthology.orchestra.run.smithi164.stdout:rbd-mirror-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 8.6 MB/s | 2.9 MB 00:00 2024-09-11T07:24:00.635 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:24:00.635 INFO:teuthology.orchestra.run.smithi164.stdout:Total 8.5 MB/s | 2.9 MB 00:00 2024-09-11T07:24:00.635 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:24:00.645 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:24:00.645 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:24:00.677 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:01:41 ago on Wed 11 Sep 2024 07:22:19 AM UTC. 2024-09-11T07:24:00.725 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:24:00.725 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:24:00.814 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-11T07:24:00.815 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:24:00.815 INFO:teuthology.orchestra.run.smithi143.stdout: Package Architecture Version Repository Size 2024-09-11T07:24:00.815 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:24:00.815 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-11T07:24:00.815 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 165 k 2024-09-11T07:24:00.816 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:24:00.816 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-11T07:24:00.816 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-11T07:24:00.816 INFO:teuthology.orchestra.run.smithi143.stdout:Install 1 Package 2024-09-11T07:24:00.816 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:24:00.816 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 165 k 2024-09-11T07:24:00.816 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 477 k 2024-09-11T07:24:00.816 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-11T07:24:01.012 INFO:teuthology.orchestra.run.smithi143.stdout:rbd-nbd-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 847 kB/s | 165 kB 00:00 2024-09-11T07:24:01.012 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:24:01.012 INFO:teuthology.orchestra.run.smithi143.stdout:Total 839 kB/s | 165 kB 00:00 2024-09-11T07:24:01.013 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-11T07:24:01.020 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-11T07:24:01.020 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-11T07:24:01.083 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-11T07:24:01.084 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-11T07:24:01.143 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:24:01.243 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-11T07:24:01.302 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:24:01.329 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:24:01.329 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:24:01.329 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T07:24:01.329 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T07:24:01.330 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T07:24:01.330 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:24:01.351 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:24:01.777 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:24:02.054 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:24:02.055 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:24:02.055 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-11T07:24:02.055 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:24:02.055 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:24:02.055 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-11T07:24:02.324 DEBUG:teuthology.parallel:result is None 2024-09-11T07:24:02.343 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:24:02.343 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:24:02.343 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:24:02.343 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:24:02.343 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:24:02.343 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:24:02.556 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install rbd-nbd 2024-09-11T07:24:03.131 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:41 ago on Wed 11 Sep 2024 07:22:22 AM UTC. 2024-09-11T07:24:03.267 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-11T07:24:03.282 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:24:03.282 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-09-11T07:24:03.282 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:24:03.282 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-11T07:24:03.282 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 165 k 2024-09-11T07:24:03.282 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:24:03.282 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-11T07:24:03.282 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-11T07:24:03.283 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-11T07:24:03.283 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:24:03.283 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 165 k 2024-09-11T07:24:03.283 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 477 k 2024-09-11T07:24:03.283 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-11T07:24:03.480 INFO:teuthology.orchestra.run.smithi164.stdout:rbd-nbd-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 781 kB/s | 165 kB 00:00 2024-09-11T07:24:03.481 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:24:03.481 INFO:teuthology.orchestra.run.smithi164.stdout:Total 775 kB/s | 165 kB 00:00 2024-09-11T07:24:03.481 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-11T07:24:03.489 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-11T07:24:03.489 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-11T07:24:03.554 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-11T07:24:03.555 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-11T07:24:03.704 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-11T07:24:03.802 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:24:04.261 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:24:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:24:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:24:04.530 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-11T07:24:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:24:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:24:04.530 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-11T07:24:04.766 DEBUG:teuthology.parallel:result is None 2024-09-11T07:24:04.767 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:24:04.914 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T07:24:04.939 INFO:teuthology.orchestra.run.smithi143.stdout:17.2.7-1623.gd0e6828a.el9 2024-09-11T07:24:04.940 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1623.gd0e6828a.el9 2024-09-11T07:24:04.940 INFO:teuthology.task.install:The correct ceph version 17.2.7-1623.gd0e6828a is installed. 2024-09-11T07:24:04.942 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:24:05.083 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T07:24:05.106 INFO:teuthology.orchestra.run.smithi164.stdout:17.2.7-1623.gd0e6828a.el9 2024-09-11T07:24:05.106 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1623.gd0e6828a.el9 2024-09-11T07:24:05.106 INFO:teuthology.task.install:The correct ceph version 17.2.7-1623.gd0e6828a is installed. 2024-09-11T07:24:05.108 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T07:24:05.108 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-11T07:24:05.108 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T07:24:05.140 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-11T07:24:05.141 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T07:24:05.173 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T07:24:05.173 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-11T07:24:05.173 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T07:24:05.208 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T07:24:05.275 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-11T07:24:05.275 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T07:24:05.300 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T07:24:05.365 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T07:24:05.366 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-11T07:24:05.366 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T07:24:05.393 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T07:24:05.459 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-11T07:24:05.459 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T07:24:05.484 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T07:24:05.554 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T07:24:05.554 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-11T07:24:05.554 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T07:24:05.579 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T07:24:05.645 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-11T07:24:05.645 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T07:24:05.670 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T07:24:05.735 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T07:24:05.828 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'cephadm_mode': 'cephadm-package'} 2024-09-11T07:24:05.828 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:24:05.829 INFO:tasks.cephadm:Cluster fsid is d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:24:05.829 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T07:24:05.829 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.143', 'mon.c': '[v2:172.21.15.143:3301,v1:172.21.15.143:6790]', 'mon.b': '172.21.15.164'} 2024-09-11T07:24:05.829 INFO:tasks.cephadm:First mon is mon.a on smithi143 2024-09-11T07:24:05.829 INFO:tasks.cephadm:First mgr is y 2024-09-11T07:24:05.829 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T07:24:05.829 DEBUG:teuthology.orchestra.run.smithi143:> sudo hostname $(hostname -s) 2024-09-11T07:24:05.855 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname $(hostname -s) 2024-09-11T07:24:05.891 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 on all hosts... 2024-09-11T07:24:05.891 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T07:24:05.899 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T07:24:07.537 INFO:teuthology.orchestra.run.smithi143.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:24:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:24:33.932 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-11T07:24:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T07:24:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T07:24:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: "repo_digests": [ 2024-09-11T07:24:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T07:24:33.934 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-11T07:24:33.934 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-11T07:24:34.298 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-11T07:24:34.298 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T07:24:34.298 INFO:teuthology.orchestra.run.smithi143.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T07:24:34.298 INFO:teuthology.orchestra.run.smithi143.stdout: "repo_digests": [ 2024-09-11T07:24:34.298 INFO:teuthology.orchestra.run.smithi143.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T07:24:34.298 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-09-11T07:24:34.298 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-11T07:24:34.317 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph 2024-09-11T07:24:34.387 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph 2024-09-11T07:24:34.416 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 777 /etc/ceph 2024-09-11T07:24:34.441 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /etc/ceph 2024-09-11T07:24:34.481 INFO:tasks.cephadm:Writing seed config... 2024-09-11T07:24:34.482 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T07:24:34.482 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T07:24:34.483 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-09-11T07:24:34.483 INFO:tasks.cephadm: override: [global] ms bind msgr1 = False 2024-09-11T07:24:34.483 INFO:tasks.cephadm: override: [global] ms bind msgr2 = True 2024-09-11T07:24:34.483 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-11T07:24:34.483 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T07:24:34.483 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T07:24:34.483 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T07:24:34.483 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T07:24:34.483 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T07:24:34.483 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T07:24:34.484 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-11T07:24:34.484 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T07:24:34.499 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d3dfd926-700e-11ef-bcea-c7b262605968 mon election default strategy = 1 ms bind msgr1 = False ms bind msgr2 = True ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-11T07:24:34.500 DEBUG:teuthology.orchestra.run.smithi143:mon.a> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T07:24:34.542 DEBUG:teuthology.orchestra.run.smithi143:mgr.y> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T07:24:34.585 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T07:24:34.586 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 -v bootstrap --fsid d3dfd926-700e-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.143 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:24:34.785 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:24:34.785 INFO:teuthology.orchestra.run.smithi143.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9', '-v', 'bootstrap', '--fsid', 'd3dfd926-700e-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.143', '--skip-admin-label'] 2024-09-11T07:24:34.809 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stdout 5.2.2 2024-09-11T07:24:34.809 INFO:teuthology.orchestra.run.smithi143.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T07:24:34.810 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying podman|docker is present... 2024-09-11T07:24:34.831 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stdout 5.2.2 2024-09-11T07:24:34.832 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying lvm2 is present... 2024-09-11T07:24:34.832 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying time synchronization is in place... 2024-09-11T07:24:34.841 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T07:24:34.841 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T07:24:34.846 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T07:24:34.846 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout inactive 2024-09-11T07:24:34.853 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout enabled 2024-09-11T07:24:34.860 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout active 2024-09-11T07:24:34.860 INFO:teuthology.orchestra.run.smithi143.stdout:Unit chronyd.service is enabled and running 2024-09-11T07:24:34.860 INFO:teuthology.orchestra.run.smithi143.stdout:Repeating the final host check... 2024-09-11T07:24:34.882 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stdout 5.2.2 2024-09-11T07:24:34.882 INFO:teuthology.orchestra.run.smithi143.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T07:24:34.882 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl is present 2024-09-11T07:24:34.882 INFO:teuthology.orchestra.run.smithi143.stdout:lvcreate is present 2024-09-11T07:24:34.889 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T07:24:34.889 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T07:24:34.896 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T07:24:34.896 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout inactive 2024-09-11T07:24:34.902 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout enabled 2024-09-11T07:24:34.909 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout active 2024-09-11T07:24:34.909 INFO:teuthology.orchestra.run.smithi143.stdout:Unit chronyd.service is enabled and running 2024-09-11T07:24:34.909 INFO:teuthology.orchestra.run.smithi143.stdout:Host looks OK 2024-09-11T07:24:34.909 INFO:teuthology.orchestra.run.smithi143.stdout:Cluster fsid: d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:24:34.910 INFO:teuthology.orchestra.run.smithi143.stdout:Acquiring lock 140276424172304 on /run/cephadm/d3dfd926-700e-11ef-bcea-c7b262605968.lock 2024-09-11T07:24:34.910 INFO:teuthology.orchestra.run.smithi143.stdout:Lock 140276424172304 acquired on /run/cephadm/d3dfd926-700e-11ef-bcea-c7b262605968.lock 2024-09-11T07:24:34.910 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying IP 172.21.15.143 port 3300 ... 2024-09-11T07:24:34.911 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying IP 172.21.15.143 port 6789 ... 2024-09-11T07:24:34.911 INFO:teuthology.orchestra.run.smithi143.stdout:Base mon IP(s) is [172.21.15.143:3300, 172.21.15.143:6789], mon addrv is [v2:172.21.15.143:3300,v1:172.21.15.143:6789] 2024-09-11T07:24:34.914 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.143 metric 100 2024-09-11T07:24:34.914 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.143 metric 100 2024-09-11T07:24:34.917 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T07:24:34.917 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-11T07:24:34.917 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-11T07:24:34.920 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T07:24:34.920 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T07:24:34.920 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T07:24:34.920 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-11T07:24:34.920 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7783/64 scope link noprefixroute 2024-09-11T07:24:34.920 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T07:24:34.921 INFO:teuthology.orchestra.run.smithi143.stdout:Mon IP `172.21.15.143` is in CIDR network `172.21.0.0/20` 2024-09-11T07:24:34.921 INFO:teuthology.orchestra.run.smithi143.stdout:Mon IP `172.21.15.143` is in CIDR network `172.21.0.0/20` 2024-09-11T07:24:34.921 INFO:teuthology.orchestra.run.smithi143.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-11T07:24:34.922 INFO:teuthology.orchestra.run.smithi143.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T07:24:34.922 INFO:teuthology.orchestra.run.smithi143.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:24:35.420 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stdout 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T07:24:35.420 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:24:35.420 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T07:24:35.420 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stderr Copying blob sha256:0d2f264d63cec2f6fb0f85eed7b2c8c5b4db3ab9edd9d71a74c04647294a553f 2024-09-11T07:24:35.420 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T07:24:35.420 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stderr Copying config sha256:4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T07:24:35.420 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T07:24:36.394 INFO:teuthology.orchestra.run.smithi143.stdout:ceph: stdout ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T07:24:36.394 INFO:teuthology.orchestra.run.smithi143.stdout:Ceph version: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T07:24:36.395 INFO:teuthology.orchestra.run.smithi143.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T07:24:37.433 INFO:teuthology.orchestra.run.smithi143.stdout:stat: stdout 167 167 2024-09-11T07:24:37.433 INFO:teuthology.orchestra.run.smithi143.stdout:Creating initial keys... 2024-09-11T07:24:38.789 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph-authtool: stdout AQC1ReFmp8RnMBAA9na8nsnwhexa8L01M6M4bw== 2024-09-11T07:24:39.702 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph-authtool: stdout AQC3ReFmHLwUBxAAdYnRvpYhwGXUTn/yhajiGw== 2024-09-11T07:24:40.640 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph-authtool: stdout AQC4ReFm9OfQBBAAdBNGEsf+oSUoKort95h6fQ== 2024-09-11T07:24:40.641 INFO:teuthology.orchestra.run.smithi143.stdout:Creating initial monmap... 2024-09-11T07:24:41.654 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T07:24:41.654 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-11T07:24:41.654 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:24:41.655 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T07:24:41.655 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool for a [v2:172.21.15.143:3300,v1:172.21.15.143:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T07:24:41.655 INFO:teuthology.orchestra.run.smithi143.stdout:setting min_mon_release = octopus 2024-09-11T07:24:41.655 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/monmaptool: set fsid to d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:24:41.655 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T07:24:41.655 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:24:41.655 INFO:teuthology.orchestra.run.smithi143.stdout:Creating mon... 2024-09-11T07:24:42.560 INFO:teuthology.orchestra.run.smithi143.stdout:create mon.a on 2024-09-11T07:24:42.747 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T07:24:42.887 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T07:24:43.099 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d3dfd926-700e-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-d3dfd926-700e-11ef-bcea-c7b262605968.target. 2024-09-11T07:24:43.099 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d3dfd926-700e-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-d3dfd926-700e-11ef-bcea-c7b262605968.target. 2024-09-11T07:24:43.270 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mon.a 2024-09-11T07:24:43.271 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T07:24:43.438 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d3dfd926-700e-11ef-bcea-c7b262605968.target.wants/ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-d3dfd926-700e-11ef-bcea-c7b262605968@.service. 2024-09-11T07:24:43.926 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:43 smithi143 podman[29095]: 2024-09-11 07:24:43.889265728 +0000 UTC m=+0.333544278 container start 660f7f6b8f2d750a8076d763844671a22caa16406297cc70b7a9e2337263d9c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:24:43.999 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:24:43.999 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout disabled 2024-09-11T07:24:44.006 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:24:44.006 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout inactive 2024-09-11T07:24:44.006 INFO:teuthology.orchestra.run.smithi143.stdout:firewalld.service is not enabled 2024-09-11T07:24:44.007 INFO:teuthology.orchestra.run.smithi143.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T07:24:44.007 INFO:teuthology.orchestra.run.smithi143.stdout:Waiting for mon to start... 2024-09-11T07:24:44.007 INFO:teuthology.orchestra.run.smithi143.stdout:Waiting for mon... 2024-09-11T07:24:44.218 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:43 smithi143 ceph-mon[29133]: mkfs d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:24:44.218 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:43 smithi143 ceph-mon[29133]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T07:24:44.218 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:43 smithi143 bash[29095]: 660f7f6b8f2d750a8076d763844671a22caa16406297cc70b7a9e2337263d9c8 2024-09-11T07:24:44.218 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:43 smithi143 systemd[1]: Started Ceph mon.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:24:45.070 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T07:24:45.071 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout id: d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:24:45.071 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T07:24:45.071 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-11T07:24:45.071 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout services: 2024-09-11T07:24:45.071 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.581401s) 2024-09-11T07:24:45.071 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T07:24:45.071 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T07:24:45.071 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-11T07:24:45.071 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout data: 2024-09-11T07:24:45.071 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T07:24:45.071 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T07:24:45.071 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T07:24:45.072 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T07:24:45.072 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-11T07:24:45.072 INFO:teuthology.orchestra.run.smithi143.stdout:mon is available 2024-09-11T07:24:45.072 INFO:teuthology.orchestra.run.smithi143.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T07:24:45.200 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:44 smithi143 ceph-mon[29133]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T07:24:45.200 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:44 smithi143 ceph-mon[29133]: monmap e1: 1 mons at {a=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:24:45.201 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:44 smithi143 ceph-mon[29133]: fsmap 2024-09-11T07:24:45.201 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:44 smithi143 ceph-mon[29133]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T07:24:45.201 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:44 smithi143 ceph-mon[29133]: mgrmap e1: no daemons active 2024-09-11T07:24:45.201 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:44 smithi143 ceph-mon[29133]: from='client.? 172.21.15.143:0/2551302289' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T07:24:46.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-11T07:24:46.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T07:24:46.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout fsid = d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:24:46.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T07:24:46.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.143:3300,v1:172.21.15.143:6789] 2024-09-11T07:24:46.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T07:24:46.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T07:24:46.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T07:24:46.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T07:24:46.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-11T07:24:46.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T07:24:46.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T07:24:46.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-11T07:24:46.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T07:24:46.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T07:24:46.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T07:24:46.152 INFO:teuthology.orchestra.run.smithi143.stdout:Generating new minimal ceph.conf... 2024-09-11T07:24:46.272 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:45 smithi143 ceph-mon[29133]: from='client.? 172.21.15.143:0/3087439196' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T07:24:46.272 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:45 smithi143 ceph-mon[29133]: from='client.? 172.21.15.143:0/3087439196' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T07:24:47.210 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:46 smithi143 ceph-mon[29133]: from='client.? 172.21.15.143:0/2491327815' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:47.213 INFO:teuthology.orchestra.run.smithi143.stdout:Restarting the monitor... 2024-09-11T07:24:47.483 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:47 smithi143 systemd[1]: Stopping Ceph mon.a for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:24:47.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:47 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-a[29110]: 2024-09-11T07:24:47.480+0000 7fd3be148640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:24:47.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:47 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-a[29110]: 2024-09-11T07:24:47.480+0000 7fd3be148640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T07:24:47.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:47 smithi143 podman[29454]: 2024-09-11 07:24:47.614008328 +0000 UTC m=+0.224273532 container died 660f7f6b8f2d750a8076d763844671a22caa16406297cc70b7a9e2337263d9c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:24:48.131 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:47 smithi143 podman[29454]: 2024-09-11 07:24:47.945183811 +0000 UTC m=+0.555449014 container cleanup 660f7f6b8f2d750a8076d763844671a22caa16406297cc70b7a9e2337263d9c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T07:24:48.131 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:47 smithi143 bash[29454]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-a 2024-09-11T07:24:48.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 podman[29471]: 2024-09-11 07:24:48.129371005 +0000 UTC m=+0.512841968 container remove 660f7f6b8f2d750a8076d763844671a22caa16406297cc70b7a9e2337263d9c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T07:24:48.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T07:24:48.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 systemd[1]: Stopped Ceph mon.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:24:48.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 systemd[1]: Starting Ceph mon.a for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:24:48.850 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 podman[29566]: 2024-09-11 07:24:48.473493788 +0000 UTC m=+0.015360390 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:24:48.850 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 podman[29566]: 2024-09-11 07:24:48.578048478 +0000 UTC m=+0.119915122 container create d3f31892115fcf7c385237250325102518aa14580e3acbc8359b03863bb4e8f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-11T07:24:48.850 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 podman[29566]: 2024-09-11 07:24:48.849201171 +0000 UTC m=+0.391067772 container init d3f31892115fcf7c385237250325102518aa14580e3acbc8359b03863bb4e8f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2) 2024-09-11T07:24:48.921 INFO:teuthology.orchestra.run.smithi143.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T07:24:49.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 podman[29566]: 2024-09-11 07:24:48.853874695 +0000 UTC m=+0.395741297 container start d3f31892115fcf7c385237250325102518aa14580e3acbc8359b03863bb4e8f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-11T07:24:49.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T07:24:49.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 7 2024-09-11T07:24:49.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: pidfile_write: ignore empty --pid-file 2024-09-11T07:24:49.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: load: jerasure load: lrc 2024-09-11T07:24:49.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: RocksDB version: 6.15.5 2024-09-11T07:24:49.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T07:24:49.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Compile date Sep 10 2024 2024-09-11T07:24:49.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: DB SUMMARY 2024-09-11T07:24:49.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: DB Session ID: Q4G1IPAV7CNWU9792R46 2024-09-11T07:24:49.153 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: CURRENT file: CURRENT 2024-09-11T07:24:49.153 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T07:24:49.153 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-11T07:24:49.153 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T07:24:49.153 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 84170 ; 2024-09-11T07:24:49.153 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.error_if_exists: 0 2024-09-11T07:24:49.153 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.create_if_missing: 0 2024-09-11T07:24:49.153 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.paranoid_checks: 1 2024-09-11T07:24:49.153 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T07:24:49.153 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.env: 0x5571f0951440 2024-09-11T07:24:49.153 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.fs: Posix File System 2024-09-11T07:24:49.153 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.info_log: 0x5571f295cec0 2024-09-11T07:24:49.153 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T07:24:49.154 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.statistics: (nil) 2024-09-11T07:24:49.154 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.use_fsync: 0 2024-09-11T07:24:49.154 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_log_file_size: 0 2024-09-11T07:24:49.154 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T07:24:49.154 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T07:24:49.154 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T07:24:49.154 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T07:24:49.154 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.allow_fallocate: 1 2024-09-11T07:24:49.154 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T07:24:49.154 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T07:24:49.154 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.use_direct_reads: 0 2024-09-11T07:24:49.154 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T07:24:49.154 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T07:24:49.155 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.db_log_dir: 2024-09-11T07:24:49.155 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-11T07:24:49.155 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T07:24:49.155 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T07:24:49.155 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T07:24:49.155 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T07:24:49.155 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T07:24:49.155 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T07:24:49.155 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T07:24:49.155 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T07:24:49.155 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.write_buffer_manager: 0x5571f2a1a780 2024-09-11T07:24:49.155 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T07:24:49.155 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T07:24:49.156 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T07:24:49.156 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T07:24:49.156 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T07:24:49.156 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T07:24:49.156 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T07:24:49.156 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T07:24:49.156 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T07:24:49.156 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.unordered_write: 0 2024-09-11T07:24:49.156 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T07:24:49.156 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T07:24:49.156 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T07:24:49.156 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T07:24:49.156 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.row_cache: None 2024-09-11T07:24:49.156 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.wal_filter: None 2024-09-11T07:24:49.157 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T07:24:49.157 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T07:24:49.157 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.preserve_deletes: 0 2024-09-11T07:24:49.157 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.two_write_queues: 0 2024-09-11T07:24:49.157 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T07:24:49.157 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.atomic_flush: 0 2024-09-11T07:24:49.157 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T07:24:49.157 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T07:24:49.157 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T07:24:49.158 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.log_readahead_size: 0 2024-09-11T07:24:49.158 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T07:24:49.158 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T07:24:49.159 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T07:24:49.159 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T07:24:49.159 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T07:24:49.159 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T07:24:49.159 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_background_jobs: 2 2024-09-11T07:24:49.159 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_background_compactions: -1 2024-09-11T07:24:49.159 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_subcompactions: 1 2024-09-11T07:24:49.159 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T07:24:49.159 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T07:24:49.159 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T07:24:49.159 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T07:24:49.159 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T07:24:49.159 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T07:24:49.159 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T07:24:49.159 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T07:24:49.160 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_open_files: -1 2024-09-11T07:24:49.160 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T07:24:49.160 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T07:24:49.160 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T07:24:49.160 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T07:24:49.160 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_background_flushes: -1 2024-09-11T07:24:49.160 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Compression algorithms supported: 2024-09-11T07:24:49.160 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: kZSTD supported: 0 2024-09-11T07:24:49.160 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: kXpressCompression supported: 0 2024-09-11T07:24:49.160 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: kBZip2Compression supported: 0 2024-09-11T07:24:49.160 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T07:24:49.160 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: kLZ4Compression supported: 1 2024-09-11T07:24:49.160 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: kZlibCompression supported: 1 2024-09-11T07:24:49.161 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T07:24:49.161 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: kSnappyCompression supported: 1 2024-09-11T07:24:49.161 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T07:24:49.161 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-11T07:24:49.161 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T07:24:49.161 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T07:24:49.161 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.merge_operator: 2024-09-11T07:24:49.161 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compaction_filter: None 2024-09-11T07:24:49.161 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T07:24:49.161 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T07:24:49.161 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T07:24:49.161 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T07:24:49.161 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5571f2903a38) 2024-09-11T07:24:49.161 INFO:journalctl@ceph.mon.a.smithi143.stdout: cache_index_and_filter_blocks: 1 2024-09-11T07:24:49.161 INFO:journalctl@ceph.mon.a.smithi143.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T07:24:49.162 INFO:journalctl@ceph.mon.a.smithi143.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T07:24:49.162 INFO:journalctl@ceph.mon.a.smithi143.stdout: pin_top_level_index_and_filter: 1 2024-09-11T07:24:49.162 INFO:journalctl@ceph.mon.a.smithi143.stdout: index_type: 0 2024-09-11T07:24:49.162 INFO:journalctl@ceph.mon.a.smithi143.stdout: data_block_index_type: 0 2024-09-11T07:24:49.162 INFO:journalctl@ceph.mon.a.smithi143.stdout: index_shortening: 1 2024-09-11T07:24:49.162 INFO:journalctl@ceph.mon.a.smithi143.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T07:24:49.162 INFO:journalctl@ceph.mon.a.smithi143.stdout: hash_index_allow_collision: 1 2024-09-11T07:24:49.162 INFO:journalctl@ceph.mon.a.smithi143.stdout: checksum: 1 2024-09-11T07:24:49.162 INFO:journalctl@ceph.mon.a.smithi143.stdout: no_block_cache: 0 2024-09-11T07:24:49.162 INFO:journalctl@ceph.mon.a.smithi143.stdout: block_cache: 0x5571f29391f0 2024-09-11T07:24:49.162 INFO:journalctl@ceph.mon.a.smithi143.stdout: block_cache_name: BinnedLRUCache 2024-09-11T07:24:49.162 INFO:journalctl@ceph.mon.a.smithi143.stdout: block_cache_options: 2024-09-11T07:24:49.162 INFO:journalctl@ceph.mon.a.smithi143.stdout: capacity : 536870912 2024-09-11T07:24:49.162 INFO:journalctl@ceph.mon.a.smithi143.stdout: num_shard_bits : 4 2024-09-11T07:24:49.162 INFO:journalctl@ceph.mon.a.smithi143.stdout: strict_capacity_limit : 0 2024-09-11T07:24:49.163 INFO:journalctl@ceph.mon.a.smithi143.stdout: high_pri_pool_ratio: 0.000 2024-09-11T07:24:49.163 INFO:journalctl@ceph.mon.a.smithi143.stdout: block_cache_compressed: (nil) 2024-09-11T07:24:49.163 INFO:journalctl@ceph.mon.a.smithi143.stdout: persistent_cache: (nil) 2024-09-11T07:24:49.163 INFO:journalctl@ceph.mon.a.smithi143.stdout: block_size: 4096 2024-09-11T07:24:49.163 INFO:journalctl@ceph.mon.a.smithi143.stdout: block_size_deviation: 10 2024-09-11T07:24:49.163 INFO:journalctl@ceph.mon.a.smithi143.stdout: block_restart_interval: 16 2024-09-11T07:24:49.163 INFO:journalctl@ceph.mon.a.smithi143.stdout: index_block_restart_interval: 1 2024-09-11T07:24:49.163 INFO:journalctl@ceph.mon.a.smithi143.stdout: metadata_block_size: 4096 2024-09-11T07:24:49.163 INFO:journalctl@ceph.mon.a.smithi143.stdout: partition_filters: 0 2024-09-11T07:24:49.163 INFO:journalctl@ceph.mon.a.smithi143.stdout: use_delta_encoding: 1 2024-09-11T07:24:49.163 INFO:journalctl@ceph.mon.a.smithi143.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T07:24:49.163 INFO:journalctl@ceph.mon.a.smithi143.stdout: whole_key_filtering: 1 2024-09-11T07:24:49.163 INFO:journalctl@ceph.mon.a.smithi143.stdout: verify_compression: 0 2024-09-11T07:24:49.163 INFO:journalctl@ceph.mon.a.smithi143.stdout: read_amp_bytes_per_bit: 0 2024-09-11T07:24:49.164 INFO:journalctl@ceph.mon.a.smithi143.stdout: format_version: 4 2024-09-11T07:24:49.164 INFO:journalctl@ceph.mon.a.smithi143.stdout: enable_index_compression: 1 2024-09-11T07:24:49.164 INFO:journalctl@ceph.mon.a.smithi143.stdout: block_align: 0 2024-09-11T07:24:49.164 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T07:24:49.164 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T07:24:49.164 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compression: NoCompression 2024-09-11T07:24:49.164 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T07:24:49.164 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T07:24:49.164 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T07:24:49.164 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.num_levels: 7 2024-09-11T07:24:49.164 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T07:24:49.164 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T07:24:49.164 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T07:24:49.164 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T07:24:49.164 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T07:24:49.164 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T07:24:49.165 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T07:24:49.165 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:24:49.165 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T07:24:49.165 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T07:24:49.165 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T07:24:49.165 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T07:24:49.165 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T07:24:49.165 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T07:24:49.165 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:24:49.165 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T07:24:49.165 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T07:24:49.165 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T07:24:49.165 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T07:24:49.165 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T07:24:49.165 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T07:24:49.166 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T07:24:49.166 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T07:24:49.166 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T07:24:49.166 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T07:24:49.166 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T07:24:49.166 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T07:24:49.166 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T07:24:49.166 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T07:24:49.166 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T07:24:49.166 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T07:24:49.166 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T07:24:49.166 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T07:24:49.166 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T07:24:49.166 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T07:24:49.166 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T07:24:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T07:24:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T07:24:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T07:24:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T07:24:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T07:24:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T07:24:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T07:24:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T07:24:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T07:24:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T07:24:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T07:24:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T07:24:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T07:24:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T07:24:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.inplace_update_support: 0 2024-09-11T07:24:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T07:24:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T07:24:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T07:24:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T07:24:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.bloom_locality: 0 2024-09-11T07:24:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.max_successive_merges: 0 2024-09-11T07:24:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T07:24:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T07:24:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T07:24:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T07:24:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.ttl: 2592000 2024-09-11T07:24:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T07:24:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.enable_blob_files: false 2024-09-11T07:24:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.min_blob_size: 0 2024-09-11T07:24:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T07:24:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T07:24:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T07:24:49.169 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T07:24:49.169 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T07:24:49.169 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-11T07:24:49.169 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-11T07:24:49.169 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726039488876485, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-11T07:24:49.169 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-11T07:24:49.169 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T07:24:49.169 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726039488877209, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 80757, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 78996, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10080, "raw_average_key_size": 47, "raw_value_size": 73248, "raw_average_value_size": 343, "num_data_blocks": 10, "num_entries": 213, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726039488, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "8e3cefe4-022b-43dd-a372-1cbe59d832db", "db_session_id": "Q4G1IPAV7CNWU9792R46"}} 2024-09-11T07:24:49.169 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-11T07:24:49.169 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726039488877543, "job": 1, "event": "recovery_finished"} 2024-09-11T07:24:49.169 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T07:24:49.169 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5571f296c700 2024-09-11T07:24:49.169 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: DB pointer 0x5571f293c000 2024-09-11T07:24:49.169 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T07:24:49.170 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T07:24:49.170 INFO:journalctl@ceph.mon.a.smithi143.stdout: ** DB Stats ** 2024-09-11T07:24:49.170 INFO:journalctl@ceph.mon.a.smithi143.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:24:49.170 INFO:journalctl@ceph.mon.a.smithi143.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T07:24:49.170 INFO:journalctl@ceph.mon.a.smithi143.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T07:24:49.170 INFO:journalctl@ceph.mon.a.smithi143.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:24:49.170 INFO:journalctl@ceph.mon.a.smithi143.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T07:24:49.170 INFO:journalctl@ceph.mon.a.smithi143.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T07:24:49.170 INFO:journalctl@ceph.mon.a.smithi143.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:24:49.170 INFO:journalctl@ceph.mon.a.smithi143.stdout: 2024-09-11T07:24:49.170 INFO:journalctl@ceph.mon.a.smithi143.stdout: ** Compaction Stats [default] ** 2024-09-11T07:24:49.170 INFO:journalctl@ceph.mon.a.smithi143.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:24:49.170 INFO:journalctl@ceph.mon.a.smithi143.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:24:49.170 INFO:journalctl@ceph.mon.a.smithi143.stdout: L0 2/0 80.65 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.2 0.00 0.00 1 0.000 0 0 2024-09-11T07:24:49.170 INFO:journalctl@ceph.mon.a.smithi143.stdout: Sum 2/0 80.65 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.2 0.00 0.00 1 0.000 0 0 2024-09-11T07:24:49.171 INFO:journalctl@ceph.mon.a.smithi143.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.2 0.00 0.00 1 0.000 0 0 2024-09-11T07:24:49.171 INFO:journalctl@ceph.mon.a.smithi143.stdout: 2024-09-11T07:24:49.171 INFO:journalctl@ceph.mon.a.smithi143.stdout: ** Compaction Stats [default] ** 2024-09-11T07:24:49.171 INFO:journalctl@ceph.mon.a.smithi143.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:24:49.171 INFO:journalctl@ceph.mon.a.smithi143.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:24:49.171 INFO:journalctl@ceph.mon.a.smithi143.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 157.2 0.00 0.00 1 0.000 0 0 2024-09-11T07:24:49.171 INFO:journalctl@ceph.mon.a.smithi143.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:24:49.171 INFO:journalctl@ceph.mon.a.smithi143.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:24:49.171 INFO:journalctl@ceph.mon.a.smithi143.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:24:49.171 INFO:journalctl@ceph.mon.a.smithi143.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:24:49.171 INFO:journalctl@ceph.mon.a.smithi143.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:24:49.171 INFO:journalctl@ceph.mon.a.smithi143.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:24:49.171 INFO:journalctl@ceph.mon.a.smithi143.stdout: Cumulative compaction: 0.00 GB write, 25.06 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:24:49.171 INFO:journalctl@ceph.mon.a.smithi143.stdout: Interval compaction: 0.00 GB write, 25.06 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:24:49.171 INFO:journalctl@ceph.mon.a.smithi143.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:24:49.172 INFO:journalctl@ceph.mon.a.smithi143.stdout: 2024-09-11T07:24:49.172 INFO:journalctl@ceph.mon.a.smithi143.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:24:49.172 INFO:journalctl@ceph.mon.a.smithi143.stdout: 2024-09-11T07:24:49.172 INFO:journalctl@ceph.mon.a.smithi143.stdout: ** Compaction Stats [default] ** 2024-09-11T07:24:49.172 INFO:journalctl@ceph.mon.a.smithi143.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:24:49.172 INFO:journalctl@ceph.mon.a.smithi143.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:24:49.172 INFO:journalctl@ceph.mon.a.smithi143.stdout: L0 2/0 80.65 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.2 0.00 0.00 1 0.000 0 0 2024-09-11T07:24:49.172 INFO:journalctl@ceph.mon.a.smithi143.stdout: Sum 2/0 80.65 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.2 0.00 0.00 1 0.000 0 0 2024-09-11T07:24:49.172 INFO:journalctl@ceph.mon.a.smithi143.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T07:24:49.172 INFO:journalctl@ceph.mon.a.smithi143.stdout: 2024-09-11T07:24:49.172 INFO:journalctl@ceph.mon.a.smithi143.stdout: ** Compaction Stats [default] ** 2024-09-11T07:24:49.172 INFO:journalctl@ceph.mon.a.smithi143.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:24:49.172 INFO:journalctl@ceph.mon.a.smithi143.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:24:49.172 INFO:journalctl@ceph.mon.a.smithi143.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 157.2 0.00 0.00 1 0.000 0 0 2024-09-11T07:24:49.172 INFO:journalctl@ceph.mon.a.smithi143.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:24:49.172 INFO:journalctl@ceph.mon.a.smithi143.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:24:49.173 INFO:journalctl@ceph.mon.a.smithi143.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:24:49.173 INFO:journalctl@ceph.mon.a.smithi143.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:24:49.173 INFO:journalctl@ceph.mon.a.smithi143.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:24:49.173 INFO:journalctl@ceph.mon.a.smithi143.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:24:49.173 INFO:journalctl@ceph.mon.a.smithi143.stdout: Cumulative compaction: 0.00 GB write, 24.82 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:24:49.173 INFO:journalctl@ceph.mon.a.smithi143.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:24:49.173 INFO:journalctl@ceph.mon.a.smithi143.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:24:49.173 INFO:journalctl@ceph.mon.a.smithi143.stdout: 2024-09-11T07:24:49.173 INFO:journalctl@ceph.mon.a.smithi143.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:24:49.173 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: starting mon.a rank 0 at public addrs [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] at bind addrs [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:24:49.173 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: mon.a@-1(???) e1 preinit fsid d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:24:49.173 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: mon.a@-1(???).mds e1 new map 2024-09-11T07:24:49.173 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: mon.a@-1(???).mds e1 print_map 2024-09-11T07:24:49.174 INFO:journalctl@ceph.mon.a.smithi143.stdout: e1 2024-09-11T07:24:49.174 INFO:journalctl@ceph.mon.a.smithi143.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T07:24:49.174 INFO:journalctl@ceph.mon.a.smithi143.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T07:24:49.174 INFO:journalctl@ceph.mon.a.smithi143.stdout: legacy client fscid: -1 2024-09-11T07:24:49.174 INFO:journalctl@ceph.mon.a.smithi143.stdout: 2024-09-11T07:24:49.174 INFO:journalctl@ceph.mon.a.smithi143.stdout: No filesystems configured 2024-09-11T07:24:49.174 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T07:24:49.174 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:24:49.174 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:24:49.174 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:24:49.174 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T07:24:49.174 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 bash[29566]: d3f31892115fcf7c385237250325102518aa14580e3acbc8359b03863bb4e8f5 2024-09-11T07:24:49.174 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 systemd[1]: Started Ceph mon.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:24:49.174 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T07:24:49.174 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: monmap e1: 1 mons at {a=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:24:49.175 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: fsmap 2024-09-11T07:24:49.175 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T07:24:49.175 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:48 smithi143 ceph-mon[29604]: mgrmap e1: no daemons active 2024-09-11T07:24:49.967 INFO:teuthology.orchestra.run.smithi143.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T07:24:49.968 INFO:teuthology.orchestra.run.smithi143.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:24:49.968 INFO:teuthology.orchestra.run.smithi143.stdout:Creating mgr... 2024-09-11T07:24:49.968 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T07:24:49.968 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T07:24:50.149 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mgr.y 2024-09-11T07:24:50.149 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-11T07:24:50.291 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d3dfd926-700e-11ef-bcea-c7b262605968.target.wants/ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-d3dfd926-700e-11ef-bcea-c7b262605968@.service. 2024-09-11T07:24:50.433 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:50 smithi143 systemd[1]: Starting Ceph mgr.y for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:24:50.717 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:50 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2283816883' entity='client.admin' 2024-09-11T07:24:50.717 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:50 smithi143 podman[29840]: 2024-09-11 07:24:50.51412589 +0000 UTC m=+0.105621736 container create 413fb9952d95053b1ffe3fbd385beb1cf14bc0ef46ce518f4ae5405fed319330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-11T07:24:50.717 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:50 smithi143 podman[29840]: 2024-09-11 07:24:50.423919978 +0000 UTC m=+0.015415840 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:24:50.717 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:50 smithi143 podman[29840]: 2024-09-11 07:24:50.716172356 +0000 UTC m=+0.307668210 container init 413fb9952d95053b1ffe3fbd385beb1cf14bc0ef46ce518f4ae5405fed319330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T07:24:50.842 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:24:50.843 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout disabled 2024-09-11T07:24:50.851 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:24:50.852 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout inactive 2024-09-11T07:24:50.852 INFO:teuthology.orchestra.run.smithi143.stdout:firewalld.service is not enabled 2024-09-11T07:24:50.852 INFO:teuthology.orchestra.run.smithi143.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T07:24:50.862 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:24:50.862 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout disabled 2024-09-11T07:24:50.872 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:24:50.872 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout inactive 2024-09-11T07:24:50.872 INFO:teuthology.orchestra.run.smithi143.stdout:firewalld.service is not enabled 2024-09-11T07:24:50.872 INFO:teuthology.orchestra.run.smithi143.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T07:24:50.872 INFO:teuthology.orchestra.run.smithi143.stdout:Waiting for mgr to start... 2024-09-11T07:24:50.872 INFO:teuthology.orchestra.run.smithi143.stdout:Waiting for mgr... 2024-09-11T07:24:50.968 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:50 smithi143 podman[29840]: 2024-09-11 07:24:50.721382552 +0000 UTC m=+0.312878401 container start 413fb9952d95053b1ffe3fbd385beb1cf14bc0ef46ce518f4ae5405fed319330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:24:50.968 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:50 smithi143 bash[29840]: 413fb9952d95053b1ffe3fbd385beb1cf14bc0ef46ce518f4ae5405fed319330 2024-09-11T07:24:50.969 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:50 smithi143 systemd[1]: Started Ceph mgr.y for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:24:50.969 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:50 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:50.835+0000 7f2890fb9ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:24:50.969 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:50 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:50.901+0000 7f2890fb9ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:24:50.969 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:50 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:50.966+0000 7f2890fb9ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:24:51.240 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:51 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:51.092+0000 7f2890fb9ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:24:51.241 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:51 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:51.238+0000 7f2890fb9ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:24:51.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:51 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/4035699625' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T07:24:51.799 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:51 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:51.510+0000 7f2890fb9ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:24:51.799 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:51 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:51.571+0000 7f2890fb9ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:24:51.799 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:51 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:51.734+0000 7f2890fb9ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:24:52.054 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-11T07:24:52.054 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:24:52.054 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "fsid": "d3dfd926-700e-11ef-bcea-c7b262605968", 2024-09-11T07:24:52.054 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T07:24:52.054 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T07:24:52.054 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T07:24:52.054 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T07:24:52.054 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:24:52.054 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T07:24:52.054 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T07:24:52.054 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 0 2024-09-11T07:24:52.055 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:24:52.055 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T07:24:52.055 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T07:24:52.055 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:24:52.055 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-11T07:24:52.055 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T07:24:52.055 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:24:52.055 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T07:24:52.055 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T07:24:52.055 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:24:52.055 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T07:24:52.055 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:24:52.055 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T07:24:52.055 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T07:24:52.056 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T07:24:52.056 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T07:24:52.056 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T07:24:52.056 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T07:24:52.059 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:24:52.059 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T07:24:52.059 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T07:24:52.059 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T07:24:52.059 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T07:24:52.059 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T07:24:52.059 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T07:24:52.059 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T07:24:52.059 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T07:24:52.060 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T07:24:52.060 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:24:52.060 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T07:24:52.060 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:24:52.060 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T07:24:52.060 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T07:24:52.060 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:24:52.060 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T07:24:52.060 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T07:24:52.060 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T07:24:52.060 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T07:24:52.060 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T07:24:52.060 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T07:24:52.060 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T07:24:52.061 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:24:52.061 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:24:52.061 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:24:52.061 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T07:24:52.061 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:24:52.061 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T07:24:43.922845+0000", 2024-09-11T07:24:52.061 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:24:52.061 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:24:52.061 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T07:24:52.061 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:24:52.061 INFO:teuthology.orchestra.run.smithi143.stdout:mgr not available, waiting (1/15)... 2024-09-11T07:24:53.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:52 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:52.576+0000 7f2890fb9ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:24:53.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:52 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:52.640+0000 7f2890fb9ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:24:53.634 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:53 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:53.345+0000 7f2890fb9ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:24:53.634 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:53 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:53.454+0000 7f2890fb9ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:24:53.634 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:53 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:53.573+0000 7f2890fb9ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:24:54.010 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:53 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:53.632+0000 7f2890fb9ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:24:54.010 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:53 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:53.756+0000 7f2890fb9ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:24:54.010 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:53 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:53.821+0000 7f2890fb9ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:24:54.010 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:53 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:53.877+0000 7f2890fb9ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:24:54.010 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:54 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:54.007+0000 7f2890fb9ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:24:54.280 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:54 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:54.073+0000 7f2890fb9ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:24:54.281 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:54 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:54.140+0000 7f2890fb9ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:24:54.281 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:54 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:54.221+0000 7f2890fb9ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:24:54.281 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:54 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:54.277+0000 7f2890fb9ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:24:54.548 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:54 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:54.348+0000 7f2890fb9ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:24:54.938 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:54 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1134299277' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T07:24:55.271 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:24:55 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:24:55.130+0000 7f2890fb9ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:24:55.275 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-11T07:24:55.275 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:24:55.275 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "fsid": "d3dfd926-700e-11ef-bcea-c7b262605968", 2024-09-11T07:24:55.275 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T07:24:55.275 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T07:24:55.276 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T07:24:55.276 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T07:24:55.276 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:24:55.276 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T07:24:55.276 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T07:24:55.276 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 0 2024-09-11T07:24:55.276 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:24:55.276 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T07:24:55.276 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T07:24:55.276 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:24:55.276 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-11T07:24:55.276 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T07:24:55.276 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:24:55.277 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T07:24:55.277 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T07:24:55.277 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:24:55.277 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T07:24:55.277 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:24:55.277 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T07:24:55.277 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T07:24:55.277 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T07:24:55.277 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T07:24:55.277 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T07:24:55.277 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T07:24:55.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:24:55.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T07:24:55.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T07:24:55.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T07:24:55.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T07:24:55.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T07:24:55.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T07:24:55.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T07:24:55.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T07:24:55.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T07:24:55.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:24:55.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T07:24:55.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:24:55.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T07:24:55.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T07:24:55.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:24:55.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T07:24:55.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T07:24:55.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T07:24:55.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T07:24:55.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T07:24:55.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T07:24:55.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T07:24:55.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:24:55.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:24:55.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:24:55.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T07:24:55.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:24:55.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T07:24:43.922845+0000", 2024-09-11T07:24:55.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:24:55.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:24:55.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T07:24:55.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:24:55.283 INFO:teuthology.orchestra.run.smithi143.stdout:mgr not available, waiting (2/15)... 2024-09-11T07:24:56.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:55 smithi143 ceph-mon[29604]: Activating manager daemon y 2024-09-11T07:24:56.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:55 smithi143 ceph-mon[29604]: mgrmap e2: y(active, starting, since 0.00292692s) 2024-09-11T07:24:56.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:55 smithi143 ceph-mon[29604]: from='mgr.14100 172.21.15.143:0/3200446647' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:24:56.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:55 smithi143 ceph-mon[29604]: from='mgr.14100 172.21.15.143:0/3200446647' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:24:56.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:55 smithi143 ceph-mon[29604]: from='mgr.14100 172.21.15.143:0/3200446647' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:24:56.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:55 smithi143 ceph-mon[29604]: from='mgr.14100 172.21.15.143:0/3200446647' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:24:56.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:55 smithi143 ceph-mon[29604]: from='mgr.14100 172.21.15.143:0/3200446647' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:24:56.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:55 smithi143 ceph-mon[29604]: Manager daemon y is now available 2024-09-11T07:24:56.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:55 smithi143 ceph-mon[29604]: from='mgr.14100 172.21.15.143:0/3200446647' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:24:56.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:55 smithi143 ceph-mon[29604]: from='mgr.14100 172.21.15.143:0/3200446647' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:24:56.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:55 smithi143 ceph-mon[29604]: from='mgr.14100 172.21.15.143:0/3200446647' entity='mgr.y' 2024-09-11T07:24:56.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:55 smithi143 ceph-mon[29604]: from='mgr.14100 172.21.15.143:0/3200446647' entity='mgr.y' 2024-09-11T07:24:56.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:55 smithi143 ceph-mon[29604]: from='mgr.14100 172.21.15.143:0/3200446647' entity='mgr.y' 2024-09-11T07:24:57.404 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:57 smithi143 ceph-mon[29604]: mgrmap e3: y(active, since 1.00728s) 2024-09-11T07:24:58.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:58 smithi143 ceph-mon[29604]: mgrmap e4: y(active, since 2s) 2024-09-11T07:24:58.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:24:58 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/84909955' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T07:25:01.084 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-11T07:25:01.084 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:25:01.085 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "fsid": "d3dfd926-700e-11ef-bcea-c7b262605968", 2024-09-11T07:25:01.085 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T07:25:01.085 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T07:25:01.085 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T07:25:01.085 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T07:25:01.085 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:25:01.085 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T07:25:01.085 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T07:25:01.085 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 0 2024-09-11T07:25:01.085 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:25:01.085 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T07:25:01.085 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T07:25:01.085 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:25:01.086 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-11T07:25:01.086 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T07:25:01.086 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:25:01.086 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T07:25:01.086 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T07:25:01.086 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:25:01.086 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T07:25:01.086 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:25:01.086 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T07:25:01.086 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T07:25:01.086 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T07:25:01.086 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T07:25:01.086 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T07:25:01.086 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T07:25:01.089 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:25:01.089 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T07:25:01.090 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T07:25:01.090 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T07:25:01.090 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T07:25:01.090 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T07:25:01.090 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T07:25:01.090 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T07:25:01.090 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T07:25:01.090 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T07:25:01.090 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:25:01.090 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T07:25:01.090 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:25:01.090 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T07:25:01.090 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T07:25:01.091 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:25:01.091 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T07:25:01.091 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T07:25:01.091 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T07:25:01.091 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T07:25:01.091 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T07:25:01.091 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T07:25:01.091 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T07:25:01.091 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:25:01.092 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:25:01.092 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:25:01.092 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T07:25:01.092 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:25:01.092 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T07:24:43.922845+0000", 2024-09-11T07:25:01.092 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:25:01.092 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:25:01.092 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T07:25:01.092 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:25:01.092 INFO:teuthology.orchestra.run.smithi143.stdout:mgr is available 2024-09-11T07:25:04.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:04 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3405012717' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T07:25:10.209 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-11T07:25:10.209 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T07:25:10.209 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout fsid = d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:25:10.209 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T07:25:10.209 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T07:25:10.210 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T07:25:10.210 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T07:25:10.210 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T07:25:10.210 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-11T07:25:10.210 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T07:25:10.210 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T07:25:10.210 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-11T07:25:10.210 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T07:25:10.210 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T07:25:10.210 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T07:25:10.210 INFO:teuthology.orchestra.run.smithi143.stdout:Enabling cephadm module... 2024-09-11T07:25:11.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:10 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3077672786' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T07:25:11.299 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:10 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ignoring --setuser ceph since I am not root 2024-09-11T07:25:11.299 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:10 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ignoring --setgroup ceph since I am not root 2024-09-11T07:25:11.299 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:11.080+0000 7fc901fe5ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:25:11.299 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:11.134+0000 7fc901fe5ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:25:11.299 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:11.200+0000 7fc901fe5ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:25:11.552 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:11.326+0000 7fc901fe5ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:25:11.552 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:11.464+0000 7fc901fe5ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:25:11.826 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:11.724+0000 7fc901fe5ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:25:11.826 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:11.784+0000 7fc901fe5ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:25:12.260 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:11.954+0000 7fc901fe5ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:25:12.261 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:11 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3077672786' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T07:25:12.261 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:11 smithi143 ceph-mon[29604]: mgrmap e5: y(active, since 15s) 2024-09-11T07:25:12.909 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:12 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:12.814+0000 7fc901fe5ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:25:12.909 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:12 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:12.880+0000 7fc901fe5ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:25:12.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:25:12.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T07:25:12.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T07:25:12.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T07:25:12.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T07:25:12.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:25:12.913 INFO:teuthology.orchestra.run.smithi143.stdout:Waiting for the mgr to restart... 2024-09-11T07:25:12.913 INFO:teuthology.orchestra.run.smithi143.stdout:Waiting for mgr epoch 5... 2024-09-11T07:25:13.162 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:12 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3359028874' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T07:25:13.900 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:13 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:13.609+0000 7fc901fe5ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:25:13.900 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:13 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:13.719+0000 7fc901fe5ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:25:13.900 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:13 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:13.839+0000 7fc901fe5ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:25:13.900 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:13 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:13.897+0000 7fc901fe5ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:25:14.281 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:14.023+0000 7fc901fe5ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:25:14.281 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:14.089+0000 7fc901fe5ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:25:14.281 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:14.145+0000 7fc901fe5ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:25:14.281 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:14.279+0000 7fc901fe5ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:25:14.547 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:14.341+0000 7fc901fe5ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:25:14.547 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:14.406+0000 7fc901fe5ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:25:14.547 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:14.487+0000 7fc901fe5ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:25:14.547 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:14.544+0000 7fc901fe5ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:25:14.798 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:14.601+0000 7fc901fe5ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:25:15.799 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:15.338+0000 7fc901fe5ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:25:15.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-mon[29604]: Active manager daemon y restarted 2024-09-11T07:25:15.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-mon[29604]: Activating manager daemon y 2024-09-11T07:25:15.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-mon[29604]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T07:25:15.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-mon[29604]: mgrmap e6: y(active, starting, since 0.0032442s) 2024-09-11T07:25:15.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:25:15.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:25:15.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:25:15.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:25:15.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:25:15.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-mon[29604]: Manager daemon y is now available 2024-09-11T07:25:15.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' 2024-09-11T07:25:15.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' 2024-09-11T07:25:15.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:15.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:15.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:25:15.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:15 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:25:16.648 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:16 smithi143 ceph-mon[29604]: Found migration_current of "None". Setting to last migration. 2024-09-11T07:25:16.648 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:16 smithi143 ceph-mon[29604]: mgrmap e7: y(active, since 1.00505s) 2024-09-11T07:25:16.927 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:25:16.927 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T07:25:16.927 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T07:25:16.927 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:25:16.927 INFO:teuthology.orchestra.run.smithi143.stdout:mgr epoch 5 is available 2024-09-11T07:25:16.927 INFO:teuthology.orchestra.run.smithi143.stdout:Setting orchestrator backend to cephadm... 2024-09-11T07:25:17.178 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:17.048+0000 7fc8b43d3640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:07:25:17] ENGINE Error in HTTPServer.serve 2024-09-11T07:25:17.178 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: Traceback (most recent call last): 2024-09-11T07:25:17.178 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:25:17.178 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: self._connections.run(self.expiration_interval) 2024-09-11T07:25:17.178 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:25:17.179 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: self._run(expiration_interval) 2024-09-11T07:25:17.179 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:25:17.179 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:25:17.179 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:25:17.179 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:25:17.179 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:25:17.179 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: s = self.context.wrap_socket( 2024-09-11T07:25:17.179 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:25:17.179 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: return self.sslsocket_class._create( 2024-09-11T07:25:17.179 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:25:17.179 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: self.do_handshake() 2024-09-11T07:25:17.179 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:25:17.179 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: self._sslobj.do_handshake() 2024-09-11T07:25:17.180 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:25:17.180 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:17.502 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-mon[29604]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T07:25:17.502 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-mon[29604]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T07:25:17.502 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-mon[29604]: [11/Sep/2024:07:25:16] ENGINE Bus STARTING 2024-09-11T07:25:17.502 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' 2024-09-11T07:25:17.502 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:17 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:18.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:18 smithi143 ceph-mon[29604]: [11/Sep/2024:07:25:17] ENGINE Serving on https://172.21.15.143:7150 2024-09-11T07:25:18.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:18 smithi143 ceph-mon[29604]: [11/Sep/2024:07:25:17] ENGINE Bus STARTED 2024-09-11T07:25:18.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:18 smithi143 ceph-mon[29604]: [11/Sep/2024:07:25:17] ENGINE Error in HTTPServer.serve 2024-09-11T07:25:18.799 INFO:journalctl@ceph.mon.a.smithi143.stdout: Traceback (most recent call last): 2024-09-11T07:25:18.799 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:25:18.799 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:25:18.799 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:25:18.799 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._run(expiration_interval) 2024-09-11T07:25:18.800 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:25:18.800 INFO:journalctl@ceph.mon.a.smithi143.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:25:18.800 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:25:18.800 INFO:journalctl@ceph.mon.a.smithi143.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:25:18.800 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:25:18.800 INFO:journalctl@ceph.mon.a.smithi143.stdout: s = self.context.wrap_socket( 2024-09-11T07:25:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:25:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout: return self.sslsocket_class._create( 2024-09-11T07:25:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:25:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout: self.do_handshake() 2024-09-11T07:25:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:25:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._sslobj.do_handshake() 2024-09-11T07:25:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:25:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:18 smithi143 ceph-mon[29604]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:18 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' 2024-09-11T07:25:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:18 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:18 smithi143 ceph-mon[29604]: mgrmap e8: y(active, since 2s) 2024-09-11T07:25:19.474 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T07:25:19.474 INFO:teuthology.orchestra.run.smithi143.stdout:Generating ssh key... 2024-09-11T07:25:19.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:19 smithi143 ceph-mon[29604]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:20.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: Generating public/private rsa key pair. 2024-09-11T07:25:20.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: Your identification has been saved in /tmp/tmp1pn3ilwt/key 2024-09-11T07:25:20.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: Your public key has been saved in /tmp/tmp1pn3ilwt/key.pub 2024-09-11T07:25:20.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: The key fingerprint is: 2024-09-11T07:25:20.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: SHA256:sGvzKPs2egTNMV56d33pVQf4G7sdzB2oH9LCKAgUZf0 ceph-d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:25:20.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: The key's randomart image is: 2024-09-11T07:25:20.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: +---[RSA 3072]----+ 2024-09-11T07:25:20.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: | .oo. ... | 2024-09-11T07:25:20.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: | .. o.. . o| 2024-09-11T07:25:20.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: | . +.=. o. +| 2024-09-11T07:25:20.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: | .. =o.E. ..+oo| 2024-09-11T07:25:20.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: | ..o.S.o.o +=+| 2024-09-11T07:25:20.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: | ..o . = oo=.| 2024-09-11T07:25:20.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: | .+ . + .o.| 2024-09-11T07:25:20.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: | ..++ .. .| 2024-09-11T07:25:20.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: | oBo.. | 2024-09-11T07:25:20.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: +----[SHA256]-----+ 2024-09-11T07:25:21.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:21 smithi143 ceph-mon[29604]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:21.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:21 smithi143 ceph-mon[29604]: Generating ssh key... 2024-09-11T07:25:21.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:21 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' 2024-09-11T07:25:21.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:21 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' 2024-09-11T07:25:21.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0TMGKeKumhhwI2CC1ht8HvHd8cjlsQbXpIugEmIzDBzXHPkpFXCO2jI0vp57+80Qtugvw3bt9FvWBzrc9dgsRGUnY0CrAobRDxdCmuE8gyyO0Yw6GqZS8MGpdJkKIA5+S5D5WLXJnufdZc+42OigSyzFIcUji5/fEg4nNIPCCXsjarwRCU4+dKYa6nfC+tEqN/r+dARJDk+tk169eUae5X8cAGJCb1bzItQAQpSIxQHEvEaBUz7ZMpWBqzG8noe85JKUHRB+S1NGS5SC4Rmzzg72Ru32GDoSol5kDGJyknWw6HCaRivc/elpfvcECjyX1uFHXFUoV2SWMDwxDFyk5+lJahkr4z+eGdNbqi2lXMcp6LTrWIKyqYQD6bAJTl2FnL5TTIsxSvsNcqlglLUJ5Jn8BsyrBDzUQ61Zn0yCFfyQDeST5jSAXHwPSIFlzvIlKzxB0HbbKMFwqgAf9DWHS9aTPtrx/PsSY70uBzv+0LiJeWTGNAwoGZSXrg3dEAU8= ceph-d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:25:21.912 INFO:teuthology.orchestra.run.smithi143.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T07:25:21.913 INFO:teuthology.orchestra.run.smithi143.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T07:25:21.917 INFO:teuthology.orchestra.run.smithi143.stdout:Adding host smithi143... 2024-09-11T07:25:22.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:22 smithi143 ceph-mon[29604]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:23.525 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:23 smithi143 ceph-mon[29604]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi143", "addr": "172.21.15.143", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:24.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:24 smithi143 ceph-mon[29604]: Deploying cephadm binary to smithi143 2024-09-11T07:25:25.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout Added host 'smithi143' with addr '172.21.15.143' 2024-09-11T07:25:25.912 INFO:teuthology.orchestra.run.smithi143.stdout:Deploying unmanaged mon service... 2024-09-11T07:25:26.518 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:26 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' 2024-09-11T07:25:26.518 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:26 smithi143 ceph-mon[29604]: Added host smithi143 2024-09-11T07:25:26.518 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:26 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:27.346 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T07:25:27.346 INFO:teuthology.orchestra.run.smithi143.stdout:Deploying unmanaged mgr service... 2024-09-11T07:25:27.623 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:27 smithi143 ceph-mon[29604]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:27.623 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:27 smithi143 ceph-mon[29604]: Saving service mon spec with placement count:5 2024-09-11T07:25:27.623 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:27 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' 2024-09-11T07:25:28.605 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T07:25:29.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:29 smithi143 ceph-mon[29604]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:29.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:29 smithi143 ceph-mon[29604]: Saving service mgr spec with placement count:2 2024-09-11T07:25:29.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:29 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' 2024-09-11T07:25:30.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:30 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' 2024-09-11T07:25:30.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:30 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' 2024-09-11T07:25:30.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:30 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3829758727' entity='client.admin' 2024-09-11T07:25:31.782 INFO:teuthology.orchestra.run.smithi143.stdout:Enabling the dashboard module... 2024-09-11T07:25:32.450 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:32 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2324061235' entity='client.admin' 2024-09-11T07:25:33.438 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:33 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2710961197' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T07:25:33.438 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:33 smithi143 ceph-mon[29604]: from='mgr.14118 172.21.15.143:0/2682246271' entity='mgr.y' 2024-09-11T07:25:33.438 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:33 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ignoring --setuser ceph since I am not root 2024-09-11T07:25:33.438 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:33 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ignoring --setgroup ceph since I am not root 2024-09-11T07:25:33.438 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:33 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:33.312+0000 7f1015049ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:25:33.438 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:33 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:33.367+0000 7f1015049ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:25:33.438 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:33 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:33.436+0000 7f1015049ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:25:33.697 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:33 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:33.560+0000 7f1015049ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:25:33.697 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:33 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:33.695+0000 7f1015049ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:25:34.267 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:34 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2710961197' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T07:25:34.267 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:34 smithi143 ceph-mon[29604]: mgrmap e9: y(active, since 17s) 2024-09-11T07:25:34.267 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:33 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:33.959+0000 7f1015049ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:25:34.267 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:34 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:34.024+0000 7f1015049ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:25:34.267 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:34 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:34.193+0000 7f1015049ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:25:35.099 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:25:35.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T07:25:35.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T07:25:35.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T07:25:35.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T07:25:35.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:25:35.100 INFO:teuthology.orchestra.run.smithi143.stdout:Waiting for the mgr to restart... 2024-09-11T07:25:35.100 INFO:teuthology.orchestra.run.smithi143.stdout:Waiting for mgr epoch 9... 2024-09-11T07:25:35.356 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:35 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2452050007' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T07:25:35.356 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:35 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:35.061+0000 7f1015049ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:25:35.356 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:35 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:35.129+0000 7f1015049ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:25:35.952 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:35 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:35.841+0000 7f1015049ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:25:35.952 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:35 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:35.949+0000 7f1015049ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:25:36.251 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:36 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:36.068+0000 7f1015049ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:25:36.251 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:36 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:36.125+0000 7f1015049ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:25:36.251 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:36 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:36.249+0000 7f1015049ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:25:36.510 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:36 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:36.314+0000 7f1015049ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:25:36.511 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:36 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:36.370+0000 7f1015049ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:25:36.511 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:36 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:36.508+0000 7f1015049ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:25:36.785 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:36 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:36.573+0000 7f1015049ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:25:36.785 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:36 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:36.640+0000 7f1015049ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:25:36.785 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:36 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:36.724+0000 7f1015049ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:25:36.785 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:36 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:36.782+0000 7f1015049ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:25:37.048 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:36 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:36.843+0000 7f1015049ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:25:38.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:37 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:37.626+0000 7f1015049ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:25:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:37 smithi143 ceph-mon[29604]: Active manager daemon y restarted 2024-09-11T07:25:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:37 smithi143 ceph-mon[29604]: Activating manager daemon y 2024-09-11T07:25:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:37 smithi143 ceph-mon[29604]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T07:25:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:37 smithi143 ceph-mon[29604]: mgrmap e10: y(active, starting, since 0.00410175s) 2024-09-11T07:25:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:25:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:25:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:25:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:25:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:25:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:37 smithi143 ceph-mon[29604]: Manager daemon y is now available 2024-09-11T07:25:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:25:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:25:39.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:38.790+0000 7f0fc5ba6640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:07:25:38] ENGINE Error in HTTPServer.serve 2024-09-11T07:25:39.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: Traceback (most recent call last): 2024-09-11T07:25:39.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:25:39.050 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: self._connections.run(self.expiration_interval) 2024-09-11T07:25:39.050 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:25:39.050 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: self._run(expiration_interval) 2024-09-11T07:25:39.050 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:25:39.050 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:25:39.050 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:25:39.050 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:25:39.050 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:25:39.050 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: s = self.context.wrap_socket( 2024-09-11T07:25:39.050 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:25:39.050 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: return self.sslsocket_class._create( 2024-09-11T07:25:39.050 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:25:39.050 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: self.do_handshake() 2024-09-11T07:25:39.050 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:25:39.051 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: self._sslobj.do_handshake() 2024-09-11T07:25:39.051 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:25:39.051 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:25:38 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:25:39.319 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:25:39.319 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T07:25:39.319 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T07:25:39.320 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:25:39.320 INFO:teuthology.orchestra.run.smithi143.stdout:mgr epoch 9 is available 2024-09-11T07:25:39.320 INFO:teuthology.orchestra.run.smithi143.stdout:Generating a dashboard self-signed certificate... 2024-09-11T07:25:39.727 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:39 smithi143 ceph-mon[29604]: mgrmap e11: y(active, since 1.00565s) 2024-09-11T07:25:39.727 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:39 smithi143 ceph-mon[29604]: [11/Sep/2024:07:25:38] ENGINE Bus STARTING 2024-09-11T07:25:39.727 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:39 smithi143 ceph-mon[29604]: [11/Sep/2024:07:25:38] ENGINE Serving on https://172.21.15.143:7150 2024-09-11T07:25:39.727 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:39 smithi143 ceph-mon[29604]: [11/Sep/2024:07:25:38] ENGINE Bus STARTED 2024-09-11T07:25:39.727 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:39 smithi143 ceph-mon[29604]: [11/Sep/2024:07:25:38] ENGINE Error in HTTPServer.serve 2024-09-11T07:25:39.727 INFO:journalctl@ceph.mon.a.smithi143.stdout: Traceback (most recent call last): 2024-09-11T07:25:39.727 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:25:39.727 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:25:39.727 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:25:39.727 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._run(expiration_interval) 2024-09-11T07:25:39.728 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:25:39.728 INFO:journalctl@ceph.mon.a.smithi143.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:25:39.728 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:25:39.728 INFO:journalctl@ceph.mon.a.smithi143.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:25:39.728 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:25:39.728 INFO:journalctl@ceph.mon.a.smithi143.stdout: s = self.context.wrap_socket( 2024-09-11T07:25:39.728 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:25:39.728 INFO:journalctl@ceph.mon.a.smithi143.stdout: return self.sslsocket_class._create( 2024-09-11T07:25:39.728 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:25:39.728 INFO:journalctl@ceph.mon.a.smithi143.stdout: self.do_handshake() 2024-09-11T07:25:39.728 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:25:39.728 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._sslobj.do_handshake() 2024-09-11T07:25:39.728 INFO:journalctl@ceph.mon.a.smithi143.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:25:39.729 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:39 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:39.729 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:39 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:40.700 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T07:25:40.700 INFO:teuthology.orchestra.run.smithi143.stdout:Creating initial admin user... 2024-09-11T07:25:41.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:41 smithi143 ceph-mon[29604]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:41.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:41 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:41.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:41 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:41.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:41 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:41.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:41 smithi143 ceph-mon[29604]: mgrmap e12: y(active, since 2s) 2024-09-11T07:25:42.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:42 smithi143 ceph-mon[29604]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:42.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:42 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:42.871 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$k1RUeGWvqg55N0pNoj.zrOT1heAPd5iwXI39JArdWykBE/0RTkQb6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726039541, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T07:25:42.872 INFO:teuthology.orchestra.run.smithi143.stdout:Fetching dashboard port number... 2024-09-11T07:25:43.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:43 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/328704795' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T07:25:44.095 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T07:25:44.105 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:25:44.105 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout disabled 2024-09-11T07:25:44.113 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:25:44.113 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout inactive 2024-09-11T07:25:44.113 INFO:teuthology.orchestra.run.smithi143.stdout:firewalld.service is not enabled 2024-09-11T07:25:44.113 INFO:teuthology.orchestra.run.smithi143.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T07:25:44.115 INFO:teuthology.orchestra.run.smithi143.stdout:Ceph Dashboard is now available at: 2024-09-11T07:25:44.115 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:25:44.115 INFO:teuthology.orchestra.run.smithi143.stdout: URL: https://smithi143.front.sepia.ceph.com:8443/ 2024-09-11T07:25:44.115 INFO:teuthology.orchestra.run.smithi143.stdout: User: admin 2024-09-11T07:25:44.115 INFO:teuthology.orchestra.run.smithi143.stdout: Password: ej5su7au81 2024-09-11T07:25:44.115 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:25:44.116 INFO:teuthology.orchestra.run.smithi143.stdout:Saving cluster configuration to /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config directory 2024-09-11T07:25:44.116 INFO:teuthology.orchestra.run.smithi143.stdout:Enabling autotune for osd_memory_target 2024-09-11T07:25:45.978 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:45 smithi143 ceph-mon[29604]: mgrmap e13: y(active, since 6s) 2024-09-11T07:25:46.541 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T07:25:46.542 INFO:teuthology.orchestra.run.smithi143.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T07:25:46.542 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:25:46.542 INFO:teuthology.orchestra.run.smithi143.stdout: sudo /sbin/cephadm shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:25:46.542 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:25:46.542 INFO:teuthology.orchestra.run.smithi143.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T07:25:46.542 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:25:46.542 INFO:teuthology.orchestra.run.smithi143.stdout: sudo /sbin/cephadm shell 2024-09-11T07:25:46.542 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:25:46.542 INFO:teuthology.orchestra.run.smithi143.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T07:25:46.542 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:25:46.542 INFO:teuthology.orchestra.run.smithi143.stdout: ceph telemetry on 2024-09-11T07:25:46.542 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:25:46.542 INFO:teuthology.orchestra.run.smithi143.stdout:For more information see: 2024-09-11T07:25:46.543 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:25:46.543 INFO:teuthology.orchestra.run.smithi143.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-11T07:25:46.543 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:25:46.543 INFO:teuthology.orchestra.run.smithi143.stdout:Bootstrap complete. 2024-09-11T07:25:46.579 INFO:tasks.cephadm:Fetching config... 2024-09-11T07:25:46.579 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-11T07:25:46.580 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T07:25:46.596 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T07:25:46.596 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-11T07:25:46.596 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T07:25:46.652 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T07:25:46.652 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-11T07:25:46.652 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T07:25:46.717 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T07:25:46.717 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-11T07:25:46.717 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T07:25:46.773 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T07:25:46.773 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0TMGKeKumhhwI2CC1ht8HvHd8cjlsQbXpIugEmIzDBzXHPkpFXCO2jI0vp57+80Qtugvw3bt9FvWBzrc9dgsRGUnY0CrAobRDxdCmuE8gyyO0Yw6GqZS8MGpdJkKIA5+S5D5WLXJnufdZc+42OigSyzFIcUji5/fEg4nNIPCCXsjarwRCU4+dKYa6nfC+tEqN/r+dARJDk+tk169eUae5X8cAGJCb1bzItQAQpSIxQHEvEaBUz7ZMpWBqzG8noe85JKUHRB+S1NGS5SC4Rmzzg72Ru32GDoSol5kDGJyknWw6HCaRivc/elpfvcECjyX1uFHXFUoV2SWMDwxDFyk5+lJahkr4z+eGdNbqi2lXMcp6LTrWIKyqYQD6bAJTl2FnL5TTIsxSvsNcqlglLUJ5Jn8BsyrBDzUQ61Zn0yCFfyQDeST5jSAXHwPSIFlzvIlKzxB0HbbKMFwqgAf9DWHS9aTPtrx/PsSY70uBzv+0LiJeWTGNAwoGZSXrg3dEAU8= ceph-d3dfd926-700e-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T07:25:46.848 INFO:teuthology.orchestra.run.smithi143.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0TMGKeKumhhwI2CC1ht8HvHd8cjlsQbXpIugEmIzDBzXHPkpFXCO2jI0vp57+80Qtugvw3bt9FvWBzrc9dgsRGUnY0CrAobRDxdCmuE8gyyO0Yw6GqZS8MGpdJkKIA5+S5D5WLXJnufdZc+42OigSyzFIcUji5/fEg4nNIPCCXsjarwRCU4+dKYa6nfC+tEqN/r+dARJDk+tk169eUae5X8cAGJCb1bzItQAQpSIxQHEvEaBUz7ZMpWBqzG8noe85JKUHRB+S1NGS5SC4Rmzzg72Ru32GDoSol5kDGJyknWw6HCaRivc/elpfvcECjyX1uFHXFUoV2SWMDwxDFyk5+lJahkr4z+eGdNbqi2lXMcp6LTrWIKyqYQD6bAJTl2FnL5TTIsxSvsNcqlglLUJ5Jn8BsyrBDzUQ61Zn0yCFfyQDeST5jSAXHwPSIFlzvIlKzxB0HbbKMFwqgAf9DWHS9aTPtrx/PsSY70uBzv+0LiJeWTGNAwoGZSXrg3dEAU8= ceph-d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:25:46.857 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0TMGKeKumhhwI2CC1ht8HvHd8cjlsQbXpIugEmIzDBzXHPkpFXCO2jI0vp57+80Qtugvw3bt9FvWBzrc9dgsRGUnY0CrAobRDxdCmuE8gyyO0Yw6GqZS8MGpdJkKIA5+S5D5WLXJnufdZc+42OigSyzFIcUji5/fEg4nNIPCCXsjarwRCU4+dKYa6nfC+tEqN/r+dARJDk+tk169eUae5X8cAGJCb1bzItQAQpSIxQHEvEaBUz7ZMpWBqzG8noe85JKUHRB+S1NGS5SC4Rmzzg72Ru32GDoSol5kDGJyknWw6HCaRivc/elpfvcECjyX1uFHXFUoV2SWMDwxDFyk5+lJahkr4z+eGdNbqi2lXMcp6LTrWIKyqYQD6bAJTl2FnL5TTIsxSvsNcqlglLUJ5Jn8BsyrBDzUQ61Zn0yCFfyQDeST5jSAXHwPSIFlzvIlKzxB0HbbKMFwqgAf9DWHS9aTPtrx/PsSY70uBzv+0LiJeWTGNAwoGZSXrg3dEAU8= ceph-d3dfd926-700e-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T07:25:46.915 INFO:teuthology.orchestra.run.smithi164.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0TMGKeKumhhwI2CC1ht8HvHd8cjlsQbXpIugEmIzDBzXHPkpFXCO2jI0vp57+80Qtugvw3bt9FvWBzrc9dgsRGUnY0CrAobRDxdCmuE8gyyO0Yw6GqZS8MGpdJkKIA5+S5D5WLXJnufdZc+42OigSyzFIcUji5/fEg4nNIPCCXsjarwRCU4+dKYa6nfC+tEqN/r+dARJDk+tk169eUae5X8cAGJCb1bzItQAQpSIxQHEvEaBUz7ZMpWBqzG8noe85JKUHRB+S1NGS5SC4Rmzzg72Ru32GDoSol5kDGJyknWw6HCaRivc/elpfvcECjyX1uFHXFUoV2SWMDwxDFyk5+lJahkr4z+eGdNbqi2lXMcp6LTrWIKyqYQD6bAJTl2FnL5TTIsxSvsNcqlglLUJ5Jn8BsyrBDzUQ61Zn0yCFfyQDeST5jSAXHwPSIFlzvIlKzxB0HbbKMFwqgAf9DWHS9aTPtrx/PsSY70uBzv+0LiJeWTGNAwoGZSXrg3dEAU8= ceph-d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:25:46.929 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T07:25:47.093 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:46 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1445926768' entity='client.admin' 2024-09-11T07:25:47.179 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:25:50.045 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:50.045 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:50.045 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi143", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:50.045 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:50.045 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:50.045 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:50.045 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:49 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1886987122' entity='client.admin' 2024-09-11T07:25:50.045 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:50.045 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:50.069 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T07:25:50.070 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T07:25:50.305 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:25:52.756 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi164 2024-09-11T07:25:52.756 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-11T07:25:52.756 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.conf 2024-09-11T07:25:52.776 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-11T07:25:52.776 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:25:52.835 INFO:tasks.cephadm:Adding host smithi164 to orchestrator... 2024-09-11T07:25:52.835 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch host add smithi164 2024-09-11T07:25:53.076 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:25:53.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:53 smithi143 ceph-mon[29604]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:53.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:53 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:53.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:53 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:53.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:53 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:53.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:53 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:53.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:53 smithi143 ceph-mon[29604]: Updating smithi143:/etc/ceph/ceph.conf 2024-09-11T07:25:53.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:53 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:53.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:53 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:53.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:53 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:54.293 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:54 smithi143 ceph-mon[29604]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:25:54.294 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:54 smithi143 ceph-mon[29604]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:25:54.294 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:54 smithi143 ceph-mon[29604]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:25:55.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:55 smithi143 ceph-mon[29604]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:56.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:56 smithi143 ceph-mon[29604]: Deploying cephadm binary to smithi164 2024-09-11T07:25:57.425 INFO:teuthology.orchestra.run.smithi143.stdout:Added host 'smithi164' with addr '172.21.15.164' 2024-09-11T07:25:58.042 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T07:25:58.277 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:25:58.660 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:58 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:58.660 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:58 smithi143 ceph-mon[29604]: Added host smithi164 2024-09-11T07:25:58.660 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:58 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:58.661 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:58 smithi143 ceph-mon[29604]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:25:58.661 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:58 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:58.661 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:58 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:25:59.885 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:25:59 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:00.022 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:26:00.023 INFO:teuthology.orchestra.run.smithi143.stdout:[{"addr": "172.21.15.143", "hostname": "smithi143", "labels": [], "status": ""}, {"addr": "172.21.15.164", "hostname": "smithi164", "labels": [], "status": ""}] 2024-09-11T07:26:00.619 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T07:26:00.619 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T07:26:00.853 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:26:00.972 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:00 smithi143 ceph-mon[29604]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:00.972 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:00 smithi143 ceph-mon[29604]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:26:02.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:02 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:02.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:02 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:02.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:02 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:02.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:02 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:02.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:02 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:26:02.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:02 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:02.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:02 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:02.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:02 smithi143 ceph-mon[29604]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-11T07:26:02.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:02 smithi143 ceph-mon[29604]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:02.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:02 smithi143 ceph-mon[29604]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:26:02.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:02 smithi143 ceph-mon[29604]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:26:02.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:02 smithi143 ceph-mon[29604]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:26:02.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:02 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:02.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:02 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:02.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:02 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:adjusted tunables profile to default 2024-09-11T07:26:03.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:03 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/4046676888' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T07:26:04.191 INFO:tasks.cephadm:Adding mon.a on smithi143 2024-09-11T07:26:04.191 INFO:tasks.cephadm:Adding mon.c on smithi143 2024-09-11T07:26:04.191 INFO:tasks.cephadm:Adding mon.b on smithi164 2024-09-11T07:26:04.191 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi143:172.21.15.143=a;smithi143:[v2:172.21.15.143:3301,v1:172.21.15.143:6790]=c;smithi164:172.21.15.164=b' 2024-09-11T07:26:04.436 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:26:04.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:04 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/4046676888' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T07:26:04.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:04 smithi143 ceph-mon[29604]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:26:04.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:04 smithi143 ceph-mon[29604]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:06.126 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mon update... 2024-09-11T07:26:06.713 DEBUG:teuthology.orchestra.run.smithi143:mon.c> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T07:26:06.716 DEBUG:teuthology.orchestra.run.smithi164:mon.b> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T07:26:06.718 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T07:26:06.718 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T07:26:07.321 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:07 smithi143 ceph-mon[29604]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:07.321 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:07 smithi143 ceph-mon[29604]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi143:172.21.15.143=a;smithi143:[v2:172.21.15.143:3301,v1:172.21.15.143:6790]=c;smithi164:172.21.15.164=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:07.321 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:07 smithi143 ceph-mon[29604]: Saving service mon spec with placement smithi143:172.21.15.143=a;smithi143:[v2:172.21.15.143:3301,v1:172.21.15.143:6790]=c;smithi164:172.21.15.164=b;count:3 2024-09-11T07:26:07.321 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:07 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:07.321 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:07 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:07.321 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:07 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:07.321 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:07 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:07.321 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:07 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:07.321 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:07 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:26:07.322 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:07 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:07.322 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:07 smithi143 ceph-mon[29604]: Deploying daemon mon.b on smithi164 2024-09-11T07:26:07.676 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:26:09.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:09 smithi143 ceph-mon[29604]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:26:10.764 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":1,"fsid":"d3dfd926-700e-11ef-bcea-c7b262605968","modified":"2024-09-11T07:24:41.034132Z","created":"2024-09-11T07:24:41.034132Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T07:26:10.764 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 1 2024-09-11T07:26:11.372 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:11 smithi164 systemd[1]: Started Ceph mon.b for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:26:11.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:11 smithi143 ceph-mon[29604]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:11.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:11 smithi143 ceph-mon[29604]: from='client.? 172.21.15.164:0/432393074' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:26:12.755 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T07:26:12.755 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T07:26:13.011 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:26:16.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:26:16.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:16.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: mon.a calling monitor election 2024-09-11T07:26:16.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:16.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:16.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:16.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: mon.b calling monitor election 2024-09-11T07:26:16.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:16.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:16.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:16.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:16.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:26:16.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: monmap e2: 2 mons at {a=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0],b=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:26:16.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: fsmap 2024-09-11T07:26:16.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:26:16.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: mgrmap e13: y(active, since 38s) 2024-09-11T07:26:16.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: overall HEALTH_OK 2024-09-11T07:26:16.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:16.469 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:16.469 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:16.469 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:26:16.469 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:16 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:16.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:26:16.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:16.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: mon.a calling monitor election 2024-09-11T07:26:16.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:16.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:16.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:16.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: mon.b calling monitor election 2024-09-11T07:26:16.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:16.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:16.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:16.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:16.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:26:16.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: monmap e2: 2 mons at {a=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0],b=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:26:16.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: fsmap 2024-09-11T07:26:16.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:26:16.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: mgrmap e13: y(active, since 38s) 2024-09-11T07:26:16.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: overall HEALTH_OK 2024-09-11T07:26:16.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:16.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:16.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:16.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:26:16.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:16 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:17.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:17 smithi143 ceph-mon[29604]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:17.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:17 smithi143 ceph-mon[29604]: Deploying daemon mon.c on smithi143 2024-09-11T07:26:17.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:17 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:17.672 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:17 smithi164 ceph-mon[33167]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:17.672 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:17 smithi164 ceph-mon[33167]: Deploying daemon mon.c on smithi143 2024-09-11T07:26:17.672 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:17 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:17.801 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:26:17.801 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":2,"fsid":"d3dfd926-700e-11ef-bcea-c7b262605968","modified":"2024-09-11T07:26:11.148449Z","created":"2024-09-11T07:24:41.034132Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T07:26:17.801 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 2 2024-09-11T07:26:18.375 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:18 smithi164 ceph-mon[33167]: from='client.? 172.21.15.164:0/1270069512' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:26:18.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:18 smithi143 ceph-mon[29604]: from='client.? 172.21.15.164:0/1270069512' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:26:18.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:26:18 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:26:18.148+0000 7f0ff95db640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-11T07:26:19.224 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 podman[38413]: 2024-09-11 07:26:19.148255888 +0000 UTC m=+0.119378154 container create 98bc9ad1527a5b88433c305102834ea725e2f695b36cd934c26ebbfae4b3eafd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-09-11T07:26:19.399 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T07:26:19.399 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T07:26:19.510 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[29604]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:19.511 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 podman[38413]: 2024-09-11 07:26:19.40980737 +0000 UTC m=+0.380929637 container init 98bc9ad1527a5b88433c305102834ea725e2f695b36cd934c26ebbfae4b3eafd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2) 2024-09-11T07:26:19.511 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 podman[38413]: 2024-09-11 07:26:19.414614581 +0000 UTC m=+0.385736851 container start 98bc9ad1527a5b88433c305102834ea725e2f695b36cd934c26ebbfae4b3eafd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:26:19.511 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T07:26:19.511 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 7 2024-09-11T07:26:19.511 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: pidfile_write: ignore empty --pid-file 2024-09-11T07:26:19.512 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: load: jerasure load: lrc 2024-09-11T07:26:19.512 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: RocksDB version: 6.15.5 2024-09-11T07:26:19.512 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T07:26:19.512 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Compile date Sep 10 2024 2024-09-11T07:26:19.512 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: DB SUMMARY 2024-09-11T07:26:19.512 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: DB Session ID: SOXIT7VU72HDNCU7PHNA 2024-09-11T07:26:19.512 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: CURRENT file: CURRENT 2024-09-11T07:26:19.512 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T07:26:19.512 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-11T07:26:19.512 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-11T07:26:19.512 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-11T07:26:19.513 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.error_if_exists: 0 2024-09-11T07:26:19.513 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.create_if_missing: 0 2024-09-11T07:26:19.513 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.paranoid_checks: 1 2024-09-11T07:26:19.513 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T07:26:19.513 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.env: 0x5594b9179440 2024-09-11T07:26:19.513 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.fs: Posix File System 2024-09-11T07:26:19.513 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.info_log: 0x5594ba596ec0 2024-09-11T07:26:19.513 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T07:26:19.513 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.statistics: (nil) 2024-09-11T07:26:19.513 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.use_fsync: 0 2024-09-11T07:26:19.513 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_log_file_size: 0 2024-09-11T07:26:19.514 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T07:26:19.514 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T07:26:19.514 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T07:26:19.514 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T07:26:19.514 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.allow_fallocate: 1 2024-09-11T07:26:19.514 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T07:26:19.514 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T07:26:19.514 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.use_direct_reads: 0 2024-09-11T07:26:19.514 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T07:26:19.514 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T07:26:19.514 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.db_log_dir: 2024-09-11T07:26:19.514 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-09-11T07:26:19.515 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T07:26:19.515 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T07:26:19.515 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T07:26:19.515 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T07:26:19.515 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T07:26:19.515 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T07:26:19.515 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T07:26:19.515 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T07:26:19.515 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.write_buffer_manager: 0x5594ba6547b0 2024-09-11T07:26:19.515 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T07:26:19.515 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T07:26:19.515 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T07:26:19.515 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T07:26:19.515 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T07:26:19.515 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T07:26:19.516 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T07:26:19.516 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T07:26:19.516 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T07:26:19.516 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.unordered_write: 0 2024-09-11T07:26:19.516 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T07:26:19.516 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T07:26:19.516 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T07:26:19.516 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T07:26:19.516 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.row_cache: None 2024-09-11T07:26:19.516 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.wal_filter: None 2024-09-11T07:26:19.517 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T07:26:19.518 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T07:26:19.518 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.preserve_deletes: 0 2024-09-11T07:26:19.518 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.two_write_queues: 0 2024-09-11T07:26:19.518 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T07:26:19.518 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.atomic_flush: 0 2024-09-11T07:26:19.518 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T07:26:19.518 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T07:26:19.518 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T07:26:19.518 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.log_readahead_size: 0 2024-09-11T07:26:19.518 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T07:26:19.518 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T07:26:19.519 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T07:26:19.519 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T07:26:19.519 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T07:26:19.519 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T07:26:19.519 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_background_jobs: 2 2024-09-11T07:26:19.519 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_background_compactions: -1 2024-09-11T07:26:19.519 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_subcompactions: 1 2024-09-11T07:26:19.519 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T07:26:19.519 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T07:26:19.519 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T07:26:19.519 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T07:26:19.519 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T07:26:19.519 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T07:26:19.520 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T07:26:19.520 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T07:26:19.520 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_open_files: -1 2024-09-11T07:26:19.520 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T07:26:19.520 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T07:26:19.520 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T07:26:19.520 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T07:26:19.520 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_background_flushes: -1 2024-09-11T07:26:19.520 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Compression algorithms supported: 2024-09-11T07:26:19.520 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: kZSTD supported: 0 2024-09-11T07:26:19.520 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: kXpressCompression supported: 0 2024-09-11T07:26:19.520 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: kBZip2Compression supported: 0 2024-09-11T07:26:19.521 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T07:26:19.521 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: kLZ4Compression supported: 1 2024-09-11T07:26:19.521 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: kZlibCompression supported: 1 2024-09-11T07:26:19.521 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T07:26:19.521 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: kSnappyCompression supported: 1 2024-09-11T07:26:19.521 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T07:26:19.521 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2024-09-11T07:26:19.521 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T07:26:19.521 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T07:26:19.521 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.merge_operator: 2024-09-11T07:26:19.521 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compaction_filter: None 2024-09-11T07:26:19.521 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T07:26:19.521 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T07:26:19.521 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T07:26:19.521 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T07:26:19.522 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5594ba53da38) 2024-09-11T07:26:19.522 INFO:journalctl@ceph.mon.c.smithi143.stdout: cache_index_and_filter_blocks: 1 2024-09-11T07:26:19.522 INFO:journalctl@ceph.mon.c.smithi143.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T07:26:19.522 INFO:journalctl@ceph.mon.c.smithi143.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T07:26:19.522 INFO:journalctl@ceph.mon.c.smithi143.stdout: pin_top_level_index_and_filter: 1 2024-09-11T07:26:19.522 INFO:journalctl@ceph.mon.c.smithi143.stdout: index_type: 0 2024-09-11T07:26:19.522 INFO:journalctl@ceph.mon.c.smithi143.stdout: data_block_index_type: 0 2024-09-11T07:26:19.522 INFO:journalctl@ceph.mon.c.smithi143.stdout: index_shortening: 1 2024-09-11T07:26:19.522 INFO:journalctl@ceph.mon.c.smithi143.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T07:26:19.522 INFO:journalctl@ceph.mon.c.smithi143.stdout: hash_index_allow_collision: 1 2024-09-11T07:26:19.522 INFO:journalctl@ceph.mon.c.smithi143.stdout: checksum: 1 2024-09-11T07:26:19.522 INFO:journalctl@ceph.mon.c.smithi143.stdout: no_block_cache: 0 2024-09-11T07:26:19.522 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_cache: 0x5594ba5731f0 2024-09-11T07:26:19.522 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_cache_name: BinnedLRUCache 2024-09-11T07:26:19.523 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_cache_options: 2024-09-11T07:26:19.523 INFO:journalctl@ceph.mon.c.smithi143.stdout: capacity : 536870912 2024-09-11T07:26:19.523 INFO:journalctl@ceph.mon.c.smithi143.stdout: num_shard_bits : 4 2024-09-11T07:26:19.523 INFO:journalctl@ceph.mon.c.smithi143.stdout: strict_capacity_limit : 0 2024-09-11T07:26:19.523 INFO:journalctl@ceph.mon.c.smithi143.stdout: high_pri_pool_ratio: 0.000 2024-09-11T07:26:19.523 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_cache_compressed: (nil) 2024-09-11T07:26:19.523 INFO:journalctl@ceph.mon.c.smithi143.stdout: persistent_cache: (nil) 2024-09-11T07:26:19.523 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_size: 4096 2024-09-11T07:26:19.523 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_size_deviation: 10 2024-09-11T07:26:19.523 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_restart_interval: 16 2024-09-11T07:26:19.523 INFO:journalctl@ceph.mon.c.smithi143.stdout: index_block_restart_interval: 1 2024-09-11T07:26:19.523 INFO:journalctl@ceph.mon.c.smithi143.stdout: metadata_block_size: 4096 2024-09-11T07:26:19.523 INFO:journalctl@ceph.mon.c.smithi143.stdout: partition_filters: 0 2024-09-11T07:26:19.523 INFO:journalctl@ceph.mon.c.smithi143.stdout: use_delta_encoding: 1 2024-09-11T07:26:19.524 INFO:journalctl@ceph.mon.c.smithi143.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T07:26:19.524 INFO:journalctl@ceph.mon.c.smithi143.stdout: whole_key_filtering: 1 2024-09-11T07:26:19.524 INFO:journalctl@ceph.mon.c.smithi143.stdout: verify_compression: 0 2024-09-11T07:26:19.524 INFO:journalctl@ceph.mon.c.smithi143.stdout: read_amp_bytes_per_bit: 0 2024-09-11T07:26:19.524 INFO:journalctl@ceph.mon.c.smithi143.stdout: format_version: 4 2024-09-11T07:26:19.524 INFO:journalctl@ceph.mon.c.smithi143.stdout: enable_index_compression: 1 2024-09-11T07:26:19.524 INFO:journalctl@ceph.mon.c.smithi143.stdout: block_align: 0 2024-09-11T07:26:19.524 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T07:26:19.524 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T07:26:19.524 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compression: NoCompression 2024-09-11T07:26:19.524 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T07:26:19.524 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T07:26:19.524 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T07:26:19.524 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.num_levels: 7 2024-09-11T07:26:19.524 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T07:26:19.525 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T07:26:19.525 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T07:26:19.525 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T07:26:19.525 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T07:26:19.525 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T07:26:19.525 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T07:26:19.525 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:26:19.525 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T07:26:19.525 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T07:26:19.525 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T07:26:19.525 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T07:26:19.525 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T07:26:19.525 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T07:26:19.525 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:26:19.526 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T07:26:19.526 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T07:26:19.526 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T07:26:19.526 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T07:26:19.526 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T07:26:19.526 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T07:26:19.526 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T07:26:19.526 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T07:26:19.526 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T07:26:19.526 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T07:26:19.526 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T07:26:19.526 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T07:26:19.526 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T07:26:19.526 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T07:26:19.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T07:26:19.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T07:26:19.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T07:26:19.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T07:26:19.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T07:26:19.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T07:26:19.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T07:26:19.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T07:26:19.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T07:26:19.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T07:26:19.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T07:26:19.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T07:26:19.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T07:26:19.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T07:26:19.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T07:26:19.528 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T07:26:19.528 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T07:26:19.528 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T07:26:19.528 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T07:26:19.528 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T07:26:19.528 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T07:26:19.528 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.inplace_update_support: 0 2024-09-11T07:26:19.528 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T07:26:19.528 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T07:26:19.528 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T07:26:19.528 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T07:26:19.528 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.bloom_locality: 0 2024-09-11T07:26:19.528 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.max_successive_merges: 0 2024-09-11T07:26:19.528 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T07:26:19.528 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T07:26:19.529 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T07:26:19.529 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T07:26:19.529 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.ttl: 2592000 2024-09-11T07:26:19.529 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T07:26:19.529 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.enable_blob_files: false 2024-09-11T07:26:19.529 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.min_blob_size: 0 2024-09-11T07:26:19.529 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T07:26:19.529 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T07:26:19.529 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T07:26:19.529 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T07:26:19.529 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T07:26:19.529 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-11T07:26:19.529 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-11T07:26:19.530 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726039579437721, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-11T07:26:19.530 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-11T07:26:19.530 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T07:26:19.530 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726039579438028, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1696, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726039579, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "1b700555-0398-4b95-8a7a-73a587bf15e4", "db_session_id": "SOXIT7VU72HDNCU7PHNA"}} 2024-09-11T07:26:19.530 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-11T07:26:19.530 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726039579438333, "job": 1, "event": "recovery_finished"} 2024-09-11T07:26:19.530 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T07:26:19.530 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5594ba5a6700 2024-09-11T07:26:19.530 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: DB pointer 0x5594ba576000 2024-09-11T07:26:19.530 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T07:26:19.530 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T07:26:19.530 INFO:journalctl@ceph.mon.c.smithi143.stdout: ** DB Stats ** 2024-09-11T07:26:19.530 INFO:journalctl@ceph.mon.c.smithi143.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:26:19.530 INFO:journalctl@ceph.mon.c.smithi143.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T07:26:19.531 INFO:journalctl@ceph.mon.c.smithi143.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T07:26:19.531 INFO:journalctl@ceph.mon.c.smithi143.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:26:19.531 INFO:journalctl@ceph.mon.c.smithi143.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T07:26:19.531 INFO:journalctl@ceph.mon.c.smithi143.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T07:26:19.531 INFO:journalctl@ceph.mon.c.smithi143.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:26:19.531 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-09-11T07:26:19.531 INFO:journalctl@ceph.mon.c.smithi143.stdout: ** Compaction Stats [default] ** 2024-09-11T07:26:19.531 INFO:journalctl@ceph.mon.c.smithi143.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:26:19.531 INFO:journalctl@ceph.mon.c.smithi143.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:26:19.531 INFO:journalctl@ceph.mon.c.smithi143.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.5 0.00 0.00 1 0.000 0 0 2024-09-11T07:26:19.531 INFO:journalctl@ceph.mon.c.smithi143.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.5 0.00 0.00 1 0.000 0 0 2024-09-11T07:26:19.531 INFO:journalctl@ceph.mon.c.smithi143.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.5 0.00 0.00 1 0.000 0 0 2024-09-11T07:26:19.531 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-09-11T07:26:19.532 INFO:journalctl@ceph.mon.c.smithi143.stdout: ** Compaction Stats [default] ** 2024-09-11T07:26:19.532 INFO:journalctl@ceph.mon.c.smithi143.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:26:19.532 INFO:journalctl@ceph.mon.c.smithi143.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:26:19.532 INFO:journalctl@ceph.mon.c.smithi143.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.5 0.00 0.00 1 0.000 0 0 2024-09-11T07:26:19.532 INFO:journalctl@ceph.mon.c.smithi143.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:26:19.532 INFO:journalctl@ceph.mon.c.smithi143.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:26:19.532 INFO:journalctl@ceph.mon.c.smithi143.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:26:19.532 INFO:journalctl@ceph.mon.c.smithi143.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:26:19.532 INFO:journalctl@ceph.mon.c.smithi143.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:26:19.532 INFO:journalctl@ceph.mon.c.smithi143.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:26:19.532 INFO:journalctl@ceph.mon.c.smithi143.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:26:19.533 INFO:journalctl@ceph.mon.c.smithi143.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:26:19.533 INFO:journalctl@ceph.mon.c.smithi143.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:26:19.533 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-09-11T07:26:19.533 INFO:journalctl@ceph.mon.c.smithi143.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:26:19.533 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-09-11T07:26:19.533 INFO:journalctl@ceph.mon.c.smithi143.stdout: ** Compaction Stats [default] ** 2024-09-11T07:26:19.533 INFO:journalctl@ceph.mon.c.smithi143.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:26:19.533 INFO:journalctl@ceph.mon.c.smithi143.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:26:19.533 INFO:journalctl@ceph.mon.c.smithi143.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.5 0.00 0.00 1 0.000 0 0 2024-09-11T07:26:19.533 INFO:journalctl@ceph.mon.c.smithi143.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.5 0.00 0.00 1 0.000 0 0 2024-09-11T07:26:19.533 INFO:journalctl@ceph.mon.c.smithi143.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T07:26:19.533 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-09-11T07:26:19.533 INFO:journalctl@ceph.mon.c.smithi143.stdout: ** Compaction Stats [default] ** 2024-09-11T07:26:19.533 INFO:journalctl@ceph.mon.c.smithi143.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:26:19.534 INFO:journalctl@ceph.mon.c.smithi143.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:26:19.534 INFO:journalctl@ceph.mon.c.smithi143.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.5 0.00 0.00 1 0.000 0 0 2024-09-11T07:26:19.534 INFO:journalctl@ceph.mon.c.smithi143.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:26:19.534 INFO:journalctl@ceph.mon.c.smithi143.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:26:19.534 INFO:journalctl@ceph.mon.c.smithi143.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:26:19.534 INFO:journalctl@ceph.mon.c.smithi143.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:26:19.534 INFO:journalctl@ceph.mon.c.smithi143.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:26:19.534 INFO:journalctl@ceph.mon.c.smithi143.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:26:19.534 INFO:journalctl@ceph.mon.c.smithi143.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:26:19.534 INFO:journalctl@ceph.mon.c.smithi143.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:26:19.534 INFO:journalctl@ceph.mon.c.smithi143.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:26:19.534 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-09-11T07:26:19.534 INFO:journalctl@ceph.mon.c.smithi143.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:26:19.534 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-11T07:26:19.535 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: using public_addrv [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] 2024-09-11T07:26:19.535 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: starting mon.c rank -1 at public addrs [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] at bind addrs [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:26:19.535 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.c@-1(???) e0 preinit fsid d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:26:19.535 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.c@-1(synchronizing).mds e1 new map 2024-09-11T07:26:19.535 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-11T07:26:19.535 INFO:journalctl@ceph.mon.c.smithi143.stdout: e1 2024-09-11T07:26:19.535 INFO:journalctl@ceph.mon.c.smithi143.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T07:26:19.535 INFO:journalctl@ceph.mon.c.smithi143.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T07:26:19.535 INFO:journalctl@ceph.mon.c.smithi143.stdout: legacy client fscid: -1 2024-09-11T07:26:19.535 INFO:journalctl@ceph.mon.c.smithi143.stdout: 2024-09-11T07:26:19.535 INFO:journalctl@ceph.mon.c.smithi143.stdout: No filesystems configured 2024-09-11T07:26:19.535 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-11T07:26:19.535 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-11T07:26:19.535 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-11T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-11T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-11T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-11T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi143:172.21.15.143=a;smithi143:[v2:172.21.15.143:3301,v1:172.21.15.143:6790]=c;smithi164:172.21.15.164=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: Saving service mon spec with placement smithi143:172.21.15.143=a;smithi143:[v2:172.21.15.143:3301,v1:172.21.15.143:6790]=c;smithi164:172.21.15.164=b;count:3 2024-09-11T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: Deploying daemon mon.b on smithi164 2024-09-11T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='client.? 172.21.15.164:0/432393074' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.a calling monitor election 2024-09-11T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.b calling monitor election 2024-09-11T07:26:19.538 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:19.538 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:19.538 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:19.538 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:19.538 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:26:19.538 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: monmap e2: 2 mons at {a=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0],b=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:26:19.538 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: fsmap 2024-09-11T07:26:19.538 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:26:19.538 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mgrmap e13: y(active, since 38s) 2024-09-11T07:26:19.538 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: overall HEALTH_OK 2024-09-11T07:26:19.538 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:19.538 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:19.538 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:19.538 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:26:19.538 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:19.538 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:19.539 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: Deploying daemon mon.c on smithi143 2024-09-11T07:26:19.539 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:19.539 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: from='client.? 172.21.15.164:0/1270069512' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:26:19.539 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:19.539 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 ceph-mon[38451]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-11T07:26:19.539 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 bash[38413]: 98bc9ad1527a5b88433c305102834ea725e2f695b36cd934c26ebbfae4b3eafd 2024-09-11T07:26:19.539 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:19 smithi143 systemd[1]: Started Ceph mon.c for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:26:19.649 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:26:19.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:19 smithi164 ceph-mon[33167]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:26:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: mon.a calling monitor election 2024-09-11T07:26:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: mon.b calling monitor election 2024-09-11T07:26:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: mon.c calling monitor election 2024-09-11T07:26:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T07:26:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: monmap e3: 3 mons at {a=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0],b=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0],c=[v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:26:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: fsmap 2024-09-11T07:26:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:26:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: mgrmap e13: y(active, since 46s) 2024-09-11T07:26:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: overall HEALTH_OK 2024-09-11T07:26:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:24.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:24.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:24.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:26:24.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: mon.a calling monitor election 2024-09-11T07:26:24.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: mon.b calling monitor election 2024-09-11T07:26:24.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:24.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:24.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: mon.c calling monitor election 2024-09-11T07:26:24.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:24.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:24.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T07:26:24.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: monmap e3: 3 mons at {a=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0],b=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0],c=[v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:26:24.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: fsmap 2024-09-11T07:26:24.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:26:24.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: mgrmap e13: y(active, since 46s) 2024-09-11T07:26:24.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: overall HEALTH_OK 2024-09-11T07:26:24.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:24.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:24.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:24.803 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:24.951 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-11T07:26:24.951 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":3,"fsid":"d3dfd926-700e-11ef-bcea-c7b262605968","modified":"2024-09-11T07:26:19.461600Z","created":"2024-09-11T07:24:41.034132Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3301","nonce":0},{"type":"v1","addr":"172.21.15.143:6790","nonce":0}]},"addr":"172.21.15.143:6790/0","public_addr":"172.21.15.143:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-11T07:26:24.951 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 3 2024-09-11T07:26:24.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:26:24.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: mon.a calling monitor election 2024-09-11T07:26:24.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: mon.b calling monitor election 2024-09-11T07:26:24.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:26:24.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:24.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: mon.c calling monitor election 2024-09-11T07:26:24.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:24.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:24.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:24.953 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T07:26:24.953 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: monmap e3: 3 mons at {a=[v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0],b=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0],c=[v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:26:24.953 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: fsmap 2024-09-11T07:26:24.953 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:26:24.953 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: mgrmap e13: y(active, since 46s) 2024-09-11T07:26:24.953 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: overall HEALTH_OK 2024-09-11T07:26:24.953 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:24.953 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:24.953 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:24.953 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:25.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:25 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:25.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:25 smithi164 ceph-mon[33167]: from='client.? 172.21.15.164:0/2446318242' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:26:25.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:25 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:25.754 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T07:26:25.754 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T07:26:25.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:25 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:25.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:25 smithi143 ceph-mon[29604]: from='client.? 172.21.15.164:0/2446318242' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:26:25.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:25 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:25.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:25 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:25.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:25 smithi143 ceph-mon[38451]: from='client.? 172.21.15.164:0/2446318242' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:26:25.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:25 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:26:25.991 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:26:26.950 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[29604]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:26.950 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.950 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.950 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.950 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.950 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:26.950 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:26.950 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[29604]: Updating smithi143:/etc/ceph/ceph.conf 2024-09-11T07:26:26.950 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[29604]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-11T07:26:26.950 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.950 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.951 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.951 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.951 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.951 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[38451]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:26.951 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.951 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.951 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.951 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.951 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:26.951 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:26.951 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[38451]: Updating smithi143:/etc/ceph/ceph.conf 2024-09-11T07:26:26.951 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[38451]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-11T07:26:26.951 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.952 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.952 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.952 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.952 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:26 smithi164 ceph-mon[33167]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:26.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:26 smithi164 ceph-mon[33167]: Updating smithi143:/etc/ceph/ceph.conf 2024-09-11T07:26:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:26 smithi164 ceph-mon[33167]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-11T07:26:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:27.747 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:27 smithi143 ceph-mon[29604]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:26:27.747 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:27 smithi143 ceph-mon[29604]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:26:27.747 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:27 smithi143 ceph-mon[29604]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T07:26:27.747 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:27 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:26:27.747 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:27 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:26:27.748 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:27 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:27.748 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:27 smithi143 ceph-mon[29604]: Reconfiguring daemon mon.a on smithi143 2024-09-11T07:26:27.748 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:27 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:27.748 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:27 smithi143 ceph-mon[38451]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:26:27.748 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:27 smithi143 ceph-mon[38451]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:26:27.748 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:27 smithi143 ceph-mon[38451]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T07:26:27.748 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:27 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:26:27.748 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:27 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:26:27.748 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:27 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:27.749 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:27 smithi143 ceph-mon[38451]: Reconfiguring daemon mon.a on smithi143 2024-09-11T07:26:27.749 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:27 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:27.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:27 smithi164 ceph-mon[33167]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:26:27.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:27 smithi164 ceph-mon[33167]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:26:27.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:27 smithi164 ceph-mon[33167]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T07:26:27.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:27 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:26:27.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:27 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:26:27.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:27 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:27.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:27 smithi164 ceph-mon[33167]: Reconfiguring daemon mon.a on smithi143 2024-09-11T07:26:27.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:27 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:28.100 INFO:teuthology.orchestra.run.smithi143.stdout:# minimal ceph.conf for d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:26:28.100 INFO:teuthology.orchestra.run.smithi143.stdout:[global] 2024-09-11T07:26:28.100 INFO:teuthology.orchestra.run.smithi143.stdout: fsid = d3dfd926-700e-11ef-bcea-c7b262605968 2024-09-11T07:26:28.100 INFO:teuthology.orchestra.run.smithi143.stdout: mon_host = [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] 2024-09-11T07:26:28.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:28 smithi143 ceph-mon[29604]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:28.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:28 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2117599016' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:28.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:28 smithi143 ceph-mon[38451]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:28.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:28 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2117599016' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:28.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:28 smithi164 ceph-mon[33167]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:28.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:28 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2117599016' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:29.363 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T07:26:29.363 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-11T07:26:29.363 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T07:26:29.390 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-11T07:26:29.390 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:26:29.457 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-11T07:26:29.457 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T07:26:29.490 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-11T07:26:29.491 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:26:29.560 INFO:tasks.cephadm:Adding mgr.y on smithi143 2024-09-11T07:26:29.561 INFO:tasks.cephadm:Adding mgr.x on smithi164 2024-09-11T07:26:29.561 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi143=y;smithi164=x' 2024-09-11T07:26:29.845 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:26:29.868 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:29 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:29.868 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:29 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:29.868 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:29 smithi143 ceph-mon[29604]: Reconfiguring mon.c (monmap changed)... 2024-09-11T07:26:29.868 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:29 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:26:29.868 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:29 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:26:29.868 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:29 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:29.868 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:29 smithi143 ceph-mon[29604]: Reconfiguring daemon mon.c on smithi143 2024-09-11T07:26:29.868 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:29 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:29.869 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:29 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:29.869 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:29 smithi143 ceph-mon[38451]: Reconfiguring mon.c (monmap changed)... 2024-09-11T07:26:29.869 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:29 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:26:29.869 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:29 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:26:29.869 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:29 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:29.869 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:29 smithi143 ceph-mon[38451]: Reconfiguring daemon mon.c on smithi143 2024-09-11T07:26:30.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:29 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:30.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:29 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:30.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:29 smithi164 ceph-mon[33167]: Reconfiguring mon.c (monmap changed)... 2024-09-11T07:26:30.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:29 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:26:30.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:29 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:26:30.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:29 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:30.044 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:29 smithi164 ceph-mon[33167]: Reconfiguring daemon mon.c on smithi143 2024-09-11T07:26:30.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:30 smithi164 ceph-mon[33167]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:30.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:30 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:30.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:30 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:30.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:30 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:26:30.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:30 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:26:30.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:30 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:31.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:30 smithi143 ceph-mon[38451]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:31.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:30 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:31.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:30 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:31.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:30 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:26:31.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:30 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:26:31.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:30 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:31.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:30 smithi143 ceph-mon[29604]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:31.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:30 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:31.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:30 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:31.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:30 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:26:31.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:30 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:26:31.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:30 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:31.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:31 smithi164 ceph-mon[33167]: Reconfiguring mon.b (monmap changed)... 2024-09-11T07:26:31.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:31 smithi164 ceph-mon[33167]: Reconfiguring daemon mon.b on smithi164 2024-09-11T07:26:31.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:31 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:31.994 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mgr update... 2024-09-11T07:26:32.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:31 smithi143 ceph-mon[29604]: Reconfiguring mon.b (monmap changed)... 2024-09-11T07:26:32.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:31 smithi143 ceph-mon[29604]: Reconfiguring daemon mon.b on smithi164 2024-09-11T07:26:32.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:31 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:32.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:31 smithi143 ceph-mon[38451]: Reconfiguring mon.b (monmap changed)... 2024-09-11T07:26:32.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:31 smithi143 ceph-mon[38451]: Reconfiguring daemon mon.b on smithi164 2024-09-11T07:26:32.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:31 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:32.582 DEBUG:teuthology.orchestra.run.smithi164:mgr.x> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T07:26:32.584 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T07:26:32.585 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-11T07:26:32.585 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T07:26:32.604 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T07:26:32.605 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_1 2024-09-11T07:26:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T07:26:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:26:32.662 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-09-11T07:26:32.662 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:26:32.662 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:26:32.663 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-09-11 07:26:18.878074634 +0000 2024-09-11T07:26:32.663 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-09-11 07:21:58.179005904 +0000 2024-09-11T07:26:32.663 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-09-11 07:21:58.179005904 +0000 2024-09-11T07:26:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-09-11 07:21:58.179005904 +0000 2024-09-11T07:26:32.663 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T07:26:32.730 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-09-11T07:26:32.730 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-09-11T07:26:32.730 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000230808 s, 2.2 MB/s 2024-09-11T07:26:32.732 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T07:26:32.790 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_2 2024-09-11T07:26:32.847 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T07:26:32.847 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:26:32.847 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 979 Links: 1 2024-09-11T07:26:32.847 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:26:32.847 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:26:32.848 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-09-11 07:26:18.878074634 +0000 2024-09-11T07:26:32.848 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-09-11 07:21:58.180005702 +0000 2024-09-11T07:26:32.848 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-09-11 07:21:58.180005702 +0000 2024-09-11T07:26:32.848 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-09-11 07:21:58.180005702 +0000 2024-09-11T07:26:32.848 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T07:26:32.913 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-09-11T07:26:32.913 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-09-11T07:26:32.913 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000260179 s, 2.0 MB/s 2024-09-11T07:26:32.914 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T07:26:32.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:32 smithi164 ceph-mon[33167]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:32.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:32 smithi164 ceph-mon[33167]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi143=y;smithi164=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:32.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:32 smithi164 ceph-mon[33167]: Saving service mgr spec with placement smithi143=y;smithi164=x;count:2 2024-09-11T07:26:32.972 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_3 2024-09-11T07:26:33.028 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T07:26:33.028 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:26:33.028 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 967 Links: 1 2024-09-11T07:26:33.028 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:26:33.028 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:26:33.028 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-09-11 07:26:18.878074634 +0000 2024-09-11T07:26:33.028 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-09-11 07:21:58.179005904 +0000 2024-09-11T07:26:33.028 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-09-11 07:21:58.179005904 +0000 2024-09-11T07:26:33.028 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-09-11 07:21:58.179005904 +0000 2024-09-11T07:26:33.028 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T07:26:33.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:32 smithi143 ceph-mon[29604]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:33.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:32 smithi143 ceph-mon[29604]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi143=y;smithi164=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:33.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:32 smithi143 ceph-mon[29604]: Saving service mgr spec with placement smithi143=y;smithi164=x;count:2 2024-09-11T07:26:33.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:32 smithi143 ceph-mon[38451]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:33.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:32 smithi143 ceph-mon[38451]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi143=y;smithi164=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:33.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:32 smithi143 ceph-mon[38451]: Saving service mgr spec with placement smithi143=y;smithi164=x;count:2 2024-09-11T07:26:33.075 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-09-11T07:26:33.075 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-09-11T07:26:33.076 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000213738 s, 2.4 MB/s 2024-09-11T07:26:33.077 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T07:26:33.133 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_4 2024-09-11T07:26:33.189 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T07:26:33.189 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:26:33.189 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 983 Links: 1 2024-09-11T07:26:33.189 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:26:33.189 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:26:33.189 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-09-11 07:26:18.878074634 +0000 2024-09-11T07:26:33.189 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-09-11 07:21:58.180005702 +0000 2024-09-11T07:26:33.189 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-09-11 07:21:58.180005702 +0000 2024-09-11T07:26:33.189 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-09-11 07:21:58.180005702 +0000 2024-09-11T07:26:33.190 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T07:26:33.253 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-09-11T07:26:33.253 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-09-11T07:26:33.253 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000199488 s, 2.6 MB/s 2024-09-11T07:26:33.254 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T07:26:33.311 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-11T07:26:33.311 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T07:26:33.327 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T07:26:33.327 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-09-11T07:26:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T07:26:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:26:33.384 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-11T07:26:33.384 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:26:33.384 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:26:33.384 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-11 07:26:09.820776815 +0000 2024-09-11T07:26:33.384 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-11 07:21:58.159660931 +0000 2024-09-11T07:26:33.384 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-11 07:21:58.159660931 +0000 2024-09-11T07:26:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-11 07:21:58.159660931 +0000 2024-09-11T07:26:33.384 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T07:26:33.447 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-11T07:26:33.448 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-11T07:26:33.448 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.00020962 s, 2.4 MB/s 2024-09-11T07:26:33.449 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T07:26:33.506 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-09-11T07:26:33.562 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T07:26:33.562 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:26:33.562 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 983 Links: 1 2024-09-11T07:26:33.562 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:26:33.562 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:26:33.562 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-11 07:26:09.820776815 +0000 2024-09-11T07:26:33.562 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-11 07:21:58.160661155 +0000 2024-09-11T07:26:33.562 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-11 07:21:58.160661155 +0000 2024-09-11T07:26:33.562 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-11 07:21:58.160661155 +0000 2024-09-11T07:26:33.562 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T07:26:33.625 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-11T07:26:33.625 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-11T07:26:33.625 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000211637 s, 2.4 MB/s 2024-09-11T07:26:33.627 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T07:26:33.682 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-09-11T07:26:33.737 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T07:26:33.738 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:26:33.738 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 979 Links: 1 2024-09-11T07:26:33.738 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:26:33.738 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:26:33.738 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-11 07:26:09.820776815 +0000 2024-09-11T07:26:33.738 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-11 07:21:58.160661155 +0000 2024-09-11T07:26:33.738 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-11 07:21:58.160661155 +0000 2024-09-11T07:26:33.738 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-11 07:21:58.160661155 +0000 2024-09-11T07:26:33.738 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T07:26:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-11T07:26:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-11T07:26:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000197657 s, 2.6 MB/s 2024-09-11T07:26:33.803 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T07:26:33.858 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-09-11T07:26:33.914 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T07:26:33.914 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:26:33.914 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-11T07:26:33.914 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:26:33.914 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:26:33.914 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-11 07:26:09.821776799 +0000 2024-09-11T07:26:33.914 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-11 07:21:58.159660931 +0000 2024-09-11T07:26:33.914 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-11 07:21:58.159660931 +0000 2024-09-11T07:26:33.914 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-11 07:21:58.159660931 +0000 2024-09-11T07:26:33.914 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T07:26:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-11T07:26:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-11T07:26:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000194189 s, 2.6 MB/s 2024-09-11T07:26:33.979 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T07:26:34.035 INFO:tasks.cephadm:Deploying osd.0 on smithi143 with /dev/vg_nvme/lv_4... 2024-09-11T07:26:34.035 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-11T07:26:34.274 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:26:34.395 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:34.396 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:34.396 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:34.396 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:34.396 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:34.396 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:34.396 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:26:34.396 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T07:26:34.396 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:26:34.396 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:34.396 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[29604]: Deploying daemon mgr.x on smithi164 2024-09-11T07:26:34.396 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:34.396 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:34.397 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:34.397 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:34.397 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:34.397 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:34.397 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:26:34.397 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T07:26:34.397 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:26:34.397 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:34.397 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:34 smithi143 ceph-mon[38451]: Deploying daemon mgr.x on smithi164 2024-09-11T07:26:34.448 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:34 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:34.448 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:34 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:34.448 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:34 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:34.448 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:34 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:34.448 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:34 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:34.448 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:34 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:34.448 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:34 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:26:34.448 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:34 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T07:26:34.448 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:34 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:26:34.448 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:34 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:34.448 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:34 smithi164 ceph-mon[33167]: Deploying daemon mgr.x on smithi164 2024-09-11T07:26:35.073 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:35 smithi164 podman[35217]: 2024-09-11 07:26:35.03948038 +0000 UTC m=+0.125725247 container create 5b741da46660f56896d976b3c7181be804af67e0f87abfa2c200648b1a793183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731) 2024-09-11T07:26:35.334 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:35 smithi164 ceph-mon[33167]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:35.335 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:35 smithi164 podman[35217]: 2024-09-11 07:26:35.233781757 +0000 UTC m=+0.320026628 container init 5b741da46660f56896d976b3c7181be804af67e0f87abfa2c200648b1a793183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T07:26:35.335 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:35 smithi164 podman[35217]: 2024-09-11 07:26:35.238459676 +0000 UTC m=+0.324704543 container start 5b741da46660f56896d976b3c7181be804af67e0f87abfa2c200648b1a793183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-11T07:26:35.335 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:35 smithi164 ceph-mgr[35255]: -- 172.21.15.164:0/166478335 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558959f92340 con 0x558959f9f000 2024-09-11T07:26:35.369 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:35 smithi143 ceph-mon[29604]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:35.370 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:35 smithi143 ceph-mon[38451]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:35.633 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:35 smithi164 bash[35217]: 5b741da46660f56896d976b3c7181be804af67e0f87abfa2c200648b1a793183 2024-09-11T07:26:35.633 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:35 smithi164 systemd[1]: Started Ceph mgr.x for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:26:35.633 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:35 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:35.355+0000 7fba89e0dec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:26:35.633 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:35 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:35.428+0000 7fba89e0dec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:26:35.633 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:35 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:35.496+0000 7fba89e0dec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:26:35.633 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:35 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:35.631+0000 7fba89e0dec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:26:35.966 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:35 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:35.785+0000 7fba89e0dec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:26:36.037 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-11T07:26:36.119 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-11T07:26:36.133 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-09-11T07:26:36.133 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-09-11T07:26:36.134 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129066 s, 812 MB/s 2024-09-11T07:26:36.134 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-09-11T07:26:36.404 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:36 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:36.059+0000 7fba89e0dec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:26:36.404 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:36 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:36.125+0000 7fba89e0dec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:26:36.404 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:36 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:36.295+0000 7fba89e0dec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:26:36.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:36 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:36.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:36 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:36.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:36 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:36.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:36 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:36.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:36 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:36.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:36 smithi164 ceph-mon[33167]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:36.777 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:36 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:36.777 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:36 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:36.777 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:36 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:36.777 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:36 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:36.778 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:36 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:36.778 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:36 smithi143 ceph-mon[29604]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:36.778 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:36 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:36.778 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:36 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:36.778 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:36 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:36.778 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:36 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:36.778 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:36 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:36.778 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:36 smithi143 ceph-mon[38451]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:36.802 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_4 2024-09-11T07:26:37.466 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:37 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:37.157+0000 7fba89e0dec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:26:37.466 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:37 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:37.221+0000 7fba89e0dec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:26:37.653 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:26:37.882 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:37.882 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:37.882 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:37.882 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:37.882 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:37.882 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[29604]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T07:26:37.882 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:26:37.882 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:26:37.883 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:37.883 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[29604]: Reconfiguring daemon mgr.y on smithi143 2024-09-11T07:26:37.883 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:37.883 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:37.883 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:37.883 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:37.883 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:37.883 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[38451]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T07:26:37.883 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:26:37.883 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:26:37.883 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:37.883 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:37 smithi143 ceph-mon[38451]: Reconfiguring daemon mgr.y on smithi143 2024-09-11T07:26:37.940 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:37.940 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:37.940 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:37.940 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:37.941 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:37.941 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:37 smithi164 ceph-mon[33167]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T07:26:37.941 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:26:37.941 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:26:37.941 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:37.941 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:37 smithi164 ceph-mon[33167]: Reconfiguring daemon mgr.y on smithi143 2024-09-11T07:26:38.216 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:37 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:37.938+0000 7fba89e0dec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:26:38.217 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:38.050+0000 7fba89e0dec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:26:38.217 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:38.171+0000 7fba89e0dec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:26:38.488 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:38.231+0000 7fba89e0dec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:26:38.489 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:38.360+0000 7fba89e0dec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:26:38.489 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:38.428+0000 7fba89e0dec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:26:38.489 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:38.486+0000 7fba89e0dec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:26:38.753 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:38.620+0000 7fba89e0dec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:26:38.753 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:38.685+0000 7fba89e0dec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:26:38.753 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:38.751+0000 7fba89e0dec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:26:38.753 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-mon[33167]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:38.753 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:38.753 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:38.753 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:38.753 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:38.753 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:38.754 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:38.754 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:38.762 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:38 smithi143 ceph-mon[38451]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:38.762 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:38 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:38.762 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:38 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:38.762 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:38 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:38.762 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:38 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:38.762 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:38 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:38.762 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:38 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:38.762 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:38 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:38.763 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:38 smithi143 ceph-mon[29604]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:38.763 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:38 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:38.763 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:38 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:38.763 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:38 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:38.763 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:38 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:38.763 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:38 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:38.763 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:38 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:38.763 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:38 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:39.216 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:38.831+0000 7fba89e0dec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:26:39.216 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:38.887+0000 7fba89e0dec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:26:39.217 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:38 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:38.945+0000 7fba89e0dec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:26:39.967 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:26:39 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:26:39.692+0000 7fba89e0dec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:26:40.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:40 smithi164 ceph-mon[33167]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:40.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:40 smithi164 ceph-mon[33167]: Standby manager daemon x started 2024-09-11T07:26:40.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:40 smithi164 ceph-mon[33167]: from='mgr.? 172.21.15.164:0/811089912' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:26:40.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:40 smithi164 ceph-mon[33167]: from='mgr.? 172.21.15.164:0/811089912' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:26:40.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:40 smithi164 ceph-mon[33167]: from='mgr.? 172.21.15.164:0/811089912' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:26:40.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:40 smithi164 ceph-mon[33167]: from='mgr.? 172.21.15.164:0/811089912' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:26:40.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:40 smithi164 ceph-mon[33167]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:40.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:40 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:26:40.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:40 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:26:40.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:40 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:41.042 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[29604]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:41.042 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[29604]: Standby manager daemon x started 2024-09-11T07:26:41.042 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[29604]: from='mgr.? 172.21.15.164:0/811089912' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:26:41.042 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[29604]: from='mgr.? 172.21.15.164:0/811089912' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:26:41.042 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[29604]: from='mgr.? 172.21.15.164:0/811089912' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:26:41.042 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[29604]: from='mgr.? 172.21.15.164:0/811089912' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:26:41.042 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[29604]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:41.042 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:26:41.042 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:26:41.042 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:41.042 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[38451]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:41.043 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[38451]: Standby manager daemon x started 2024-09-11T07:26:41.043 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[38451]: from='mgr.? 172.21.15.164:0/811089912' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:26:41.043 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[38451]: from='mgr.? 172.21.15.164:0/811089912' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:26:41.043 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[38451]: from='mgr.? 172.21.15.164:0/811089912' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:26:41.043 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[38451]: from='mgr.? 172.21.15.164:0/811089912' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:26:41.043 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[38451]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:41.043 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:26:41.043 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:26:41.043 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:40 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:41.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:41 smithi164 ceph-mon[33167]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-11T07:26:41.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:41 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:26:42.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:41 smithi143 ceph-mon[29604]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-11T07:26:42.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:41 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:26:42.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:41 smithi143 ceph-mon[38451]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-11T07:26:42.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:41 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:26:42.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:42 smithi143 ceph-mon[29604]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:42.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:42 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2657151505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aafa4449-5343-4ce4-bb8a-1935f5e4fcbd"}]: dispatch 2024-09-11T07:26:42.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:42 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2657151505' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aafa4449-5343-4ce4-bb8a-1935f5e4fcbd"}]': finished 2024-09-11T07:26:42.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:42 smithi143 ceph-mon[29604]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T07:26:42.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:42 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:26:42.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:42 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1755257032' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:26:42.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:42 smithi143 ceph-mon[38451]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:42.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:42 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2657151505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aafa4449-5343-4ce4-bb8a-1935f5e4fcbd"}]: dispatch 2024-09-11T07:26:42.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:42 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2657151505' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aafa4449-5343-4ce4-bb8a-1935f5e4fcbd"}]': finished 2024-09-11T07:26:42.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:42 smithi143 ceph-mon[38451]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T07:26:42.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:42 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:26:42.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:42 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1755257032' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:26:42.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:42 smithi164 ceph-mon[33167]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:42.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:42 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2657151505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aafa4449-5343-4ce4-bb8a-1935f5e4fcbd"}]: dispatch 2024-09-11T07:26:42.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:42 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2657151505' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aafa4449-5343-4ce4-bb8a-1935f5e4fcbd"}]': finished 2024-09-11T07:26:42.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:42 smithi164 ceph-mon[33167]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T07:26:42.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:42 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:26:42.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:42 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1755257032' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:26:44.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:44 smithi164 ceph-mon[33167]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:45.008 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:44 smithi143 ceph-mon[29604]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:45.009 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:44 smithi143 ceph-mon[38451]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:46.875 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:46 smithi143 ceph-mon[29604]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:46.875 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:46 smithi143 ceph-mon[38451]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:46.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:46 smithi164 ceph-mon[33167]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:48.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:48 smithi164 ceph-mon[33167]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:48.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:48 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T07:26:48.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:48 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:48.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:48 smithi164 ceph-mon[33167]: Deploying daemon osd.0 on smithi143 2024-09-11T07:26:49.013 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:48 smithi143 ceph-mon[29604]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:49.013 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:48 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T07:26:49.013 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:48 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:49.013 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:48 smithi143 ceph-mon[29604]: Deploying daemon osd.0 on smithi143 2024-09-11T07:26:49.014 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:48 smithi143 ceph-mon[38451]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:49.014 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:48 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T07:26:49.014 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:48 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:49.014 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:48 smithi143 ceph-mon[38451]: Deploying daemon osd.0 on smithi143 2024-09-11T07:26:50.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:50 smithi143 ceph-mon[29604]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:50.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:50 smithi143 ceph-mon[38451]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:50.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:50 smithi164 ceph-mon[33167]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:52.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:52 smithi143 ceph-mon[29604]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:52.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:52 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:52.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:52 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:52.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:52 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:52.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:52 smithi143 ceph-mon[38451]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:52.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:52 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:52.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:52 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:52.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:52 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:52.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:52 smithi164 ceph-mon[33167]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:52.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:52 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:52.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:52 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:52.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:52 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:54.822 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:54 smithi143 ceph-mon[29604]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:54.822 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:54 smithi143 ceph-mon[38451]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:54.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:54 smithi164 ceph-mon[33167]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:55.883 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 0 on host 'smithi143' 2024-09-11T07:26:56.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:56 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:56.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:56 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:56.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:56 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:56.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:56 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:56.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:56 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:56.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:56 smithi164 ceph-mon[33167]: from='osd.0 v2:172.21.15.143:6801/168444810' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:26:56.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:56 smithi164 ceph-mon[33167]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:26:56.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:56 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:56.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:56 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:56.469 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:56 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:56.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:56.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:56.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:56.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:56.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:56.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[29604]: from='osd.0 v2:172.21.15.143:6801/168444810' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:26:56.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[29604]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:26:56.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:56.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:56.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:56.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:56.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:56.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:56.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:56.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:56.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[38451]: from='osd.0 v2:172.21.15.143:6801/168444810' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:26:56.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[38451]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:26:56.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:56.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:56.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:56 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:56.577 DEBUG:teuthology.orchestra.run.smithi143:osd.0> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T07:26:56.579 INFO:tasks.cephadm:Deploying osd.1 on smithi143 with /dev/vg_nvme/lv_3... 2024-09-11T07:26:56.579 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-11T07:26:57.306 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:26:57.418 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:57 smithi143 ceph-mon[38451]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:57.474 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:57 smithi143 ceph-mon[38451]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T07:26:57.475 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:57 smithi143 ceph-mon[38451]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T07:26:57.475 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:57 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:26:57.475 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:57 smithi143 ceph-mon[38451]: from='osd.0 v2:172.21.15.143:6801/168444810' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:26:57.475 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:57 smithi143 ceph-mon[38451]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:26:57.475 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:57 smithi143 ceph-mon[29604]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:57.475 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:57 smithi143 ceph-mon[29604]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T07:26:57.475 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:57 smithi143 ceph-mon[29604]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T07:26:57.475 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:57 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:26:57.476 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:57 smithi143 ceph-mon[29604]: from='osd.0 v2:172.21.15.143:6801/168444810' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:26:57.476 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:57 smithi143 ceph-mon[29604]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:26:57.476 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:57 smithi164 ceph-mon[33167]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:57.476 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:57 smithi164 ceph-mon[33167]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T07:26:57.476 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:57 smithi164 ceph-mon[33167]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T07:26:57.476 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:57 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:26:57.476 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:57 smithi164 ceph-mon[33167]: from='osd.0 v2:172.21.15.143:6801/168444810' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:26:57.476 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:57 smithi164 ceph-mon[33167]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:26:57.685 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:26:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-0[42706]: 2024-09-11T07:26:57.603+0000 7f272f065640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T07:26:57.685 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:26:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-0[42706]: 2024-09-11T07:26:57.605+0000 7f272969b640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:26:58.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:58 smithi143 ceph-mon[29604]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-11T07:26:58.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:58 smithi143 ceph-mon[29604]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T07:26:58.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:58 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:26:58.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:58 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:26:58.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:58 smithi143 ceph-mon[29604]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:58.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:58 smithi143 ceph-mon[38451]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-11T07:26:58.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:58 smithi143 ceph-mon[38451]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T07:26:58.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:58 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:26:58.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:58 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:26:58.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:58 smithi143 ceph-mon[38451]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:58.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:58 smithi164 ceph-mon[33167]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-11T07:26:58.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:58 smithi164 ceph-mon[33167]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T07:26:58.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:58 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:26:58.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:58 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:26:58.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:58 smithi164 ceph-mon[33167]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:26:59.909 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[38451]: purged_snaps scrub starts 2024-09-11T07:26:59.909 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[38451]: purged_snaps scrub ok 2024-09-11T07:26:59.909 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[38451]: osd.0 v2:172.21.15.143:6801/168444810 boot 2024-09-11T07:26:59.909 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[38451]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T07:26:59.909 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:26:59.909 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:59.909 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:59.909 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:26:59.909 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:59.909 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:59.910 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:59.910 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:59.910 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[29604]: purged_snaps scrub starts 2024-09-11T07:26:59.910 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[29604]: purged_snaps scrub ok 2024-09-11T07:26:59.910 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[29604]: osd.0 v2:172.21.15.143:6801/168444810 boot 2024-09-11T07:26:59.910 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[29604]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T07:26:59.910 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:26:59.910 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:59.910 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:59.910 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:26:59.910 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:59.910 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:59.910 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:59.911 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:26:59 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:59.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:59 smithi164 ceph-mon[33167]: purged_snaps scrub starts 2024-09-11T07:26:59.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:59 smithi164 ceph-mon[33167]: purged_snaps scrub ok 2024-09-11T07:26:59.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:59 smithi164 ceph-mon[33167]: osd.0 v2:172.21.15.143:6801/168444810 boot 2024-09-11T07:26:59.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:59 smithi164 ceph-mon[33167]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T07:26:59.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:59 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:26:59.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:59 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:59.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:59 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:59.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:59 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:26:59.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:59 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:26:59.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:59 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:59.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:59 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:59.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:26:59 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:00.333 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-11T07:27:00.419 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-11T07:27:00.434 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-09-11T07:27:00.434 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-09-11T07:27:00.434 INFO:teuthology.orchestra.run.smithi143.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128269 s, 817 MB/s 2024-09-11T07:27:00.434 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-09-11T07:27:00.739 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:00 smithi143 ceph-mon[38451]: Detected new or changed devices on smithi143 2024-09-11T07:27:00.740 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:00 smithi143 ceph-mon[38451]: Adjusting osd_memory_target on smithi143 to 16064M 2024-09-11T07:27:00.740 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:00 smithi143 ceph-mon[38451]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:00.740 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:00 smithi143 ceph-mon[29604]: Detected new or changed devices on smithi143 2024-09-11T07:27:00.740 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:00 smithi143 ceph-mon[29604]: Adjusting osd_memory_target on smithi143 to 16064M 2024-09-11T07:27:00.740 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:00 smithi143 ceph-mon[29604]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:00.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:00 smithi164 ceph-mon[33167]: Detected new or changed devices on smithi143 2024-09-11T07:27:00.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:00 smithi164 ceph-mon[33167]: Adjusting osd_memory_target on smithi143 to 16064M 2024-09-11T07:27:00.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:00 smithi164 ceph-mon[33167]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:01.014 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_3 2024-09-11T07:27:01.253 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:01.668 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:01 smithi143 ceph-mon[29604]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T07:27:01.668 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:01 smithi143 ceph-mon[38451]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T07:27:01.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:01 smithi164 ceph-mon[33167]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T07:27:02.858 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:02 smithi143 ceph-mon[29604]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:02.858 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:02 smithi143 ceph-mon[38451]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:02.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:02 smithi164 ceph-mon[33167]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:03.686 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:03 smithi143 ceph-mon[29604]: from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:27:03.686 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:03 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:27:03.687 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:03 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:27:03.687 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:03 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:03.687 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:03 smithi143 ceph-mon[38451]: from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:27:03.687 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:03 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:27:03.687 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:03 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:27:03.687 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:03 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:03.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:03 smithi164 ceph-mon[33167]: from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:27:03.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:03 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:27:03.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:03 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:27:03.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:03 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:04.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:04 smithi143 ceph-mon[29604]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:04.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:04 smithi143 ceph-mon[38451]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:04.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:04 smithi164 ceph-mon[33167]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:05.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:05 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/4107932938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "633a8442-c551-4678-bd79-d938e83c11cf"}]: dispatch 2024-09-11T07:27:05.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:05 smithi164 ceph-mon[33167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "633a8442-c551-4678-bd79-d938e83c11cf"}]: dispatch 2024-09-11T07:27:05.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:05 smithi164 ceph-mon[33167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "633a8442-c551-4678-bd79-d938e83c11cf"}]': finished 2024-09-11T07:27:05.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:05 smithi164 ceph-mon[33167]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T07:27:05.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:05 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:06.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:05 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/4107932938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "633a8442-c551-4678-bd79-d938e83c11cf"}]: dispatch 2024-09-11T07:27:06.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:05 smithi143 ceph-mon[29604]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "633a8442-c551-4678-bd79-d938e83c11cf"}]: dispatch 2024-09-11T07:27:06.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:05 smithi143 ceph-mon[29604]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "633a8442-c551-4678-bd79-d938e83c11cf"}]': finished 2024-09-11T07:27:06.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:05 smithi143 ceph-mon[29604]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T07:27:06.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:05 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:06.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:05 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/4107932938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "633a8442-c551-4678-bd79-d938e83c11cf"}]: dispatch 2024-09-11T07:27:06.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:05 smithi143 ceph-mon[38451]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "633a8442-c551-4678-bd79-d938e83c11cf"}]: dispatch 2024-09-11T07:27:06.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:05 smithi143 ceph-mon[38451]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "633a8442-c551-4678-bd79-d938e83c11cf"}]': finished 2024-09-11T07:27:06.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:05 smithi143 ceph-mon[38451]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T07:27:06.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:05 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:06.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:06 smithi164 ceph-mon[33167]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:06.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:06 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1726981364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:27:07.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:06 smithi143 ceph-mon[29604]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:07.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:06 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1726981364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:27:07.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:06 smithi143 ceph-mon[38451]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:07.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:06 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1726981364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:27:08.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:08 smithi164 ceph-mon[33167]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:09.027 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:08 smithi143 ceph-mon[29604]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:09.027 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:08 smithi143 ceph-mon[38451]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:10.921 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:10 smithi143 ceph-mon[29604]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:10.922 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:10 smithi143 ceph-mon[38451]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:10.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:10 smithi164 ceph-mon[33167]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:11.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:11 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T07:27:11.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:11 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:11.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:11 smithi164 ceph-mon[33167]: Deploying daemon osd.1 on smithi143 2024-09-11T07:27:12.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:11 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T07:27:12.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:11 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:12.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:11 smithi143 ceph-mon[29604]: Deploying daemon osd.1 on smithi143 2024-09-11T07:27:12.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:11 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T07:27:12.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:11 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:12.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:11 smithi143 ceph-mon[38451]: Deploying daemon osd.1 on smithi143 2024-09-11T07:27:12.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:12 smithi164 ceph-mon[33167]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:13.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:12 smithi143 ceph-mon[38451]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:13.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:12 smithi143 ceph-mon[29604]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:14.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:14 smithi164 ceph-mon[33167]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:15.009 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:14 smithi143 ceph-mon[29604]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:15.010 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:14 smithi143 ceph-mon[38451]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:15.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:15 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:27:15.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:15 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:15.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:15 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:15.987 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:15 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:27:15.987 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:15 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:15.987 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:15 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:15.987 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:15 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:27:15.987 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:15 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:15.987 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:15 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:16.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:16 smithi164 ceph-mon[33167]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:17.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:16 smithi143 ceph-mon[29604]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:17.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:16 smithi143 ceph-mon[38451]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:18.807 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 1 on host 'smithi143' 2024-09-11T07:27:19.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[38451]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:19.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:19.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:19.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:19.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:27:19.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:19.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[38451]: from='osd.1 v2:172.21.15.143:6805/507789371' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T07:27:19.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:27:19.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:19.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:19.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[29604]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:19.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:19.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:19.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:19.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:27:19.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:19.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[29604]: from='osd.1 v2:172.21.15.143:6805/507789371' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T07:27:19.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:27:19.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:19.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:19 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:19.397 DEBUG:teuthology.orchestra.run.smithi143:osd.1> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T07:27:19.400 INFO:tasks.cephadm:Deploying osd.2 on smithi143 with /dev/vg_nvme/lv_2... 2024-09-11T07:27:19.400 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-11T07:27:19.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:19 smithi164 ceph-mon[33167]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:19.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:19 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:19.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:19 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:19.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:19 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:19.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:19 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:27:19.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:19 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:19.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:19 smithi164 ceph-mon[33167]: from='osd.1 v2:172.21.15.143:6805/507789371' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T07:27:19.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:19 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:27:19.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:19 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:19.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:19 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:20.155 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:20.259 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:20 smithi143 ceph-mon[29604]: from='osd.1 v2:172.21.15.143:6805/507789371' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T07:27:20.259 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:20 smithi143 ceph-mon[29604]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T07:27:20.259 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:20 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:20.260 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:20 smithi143 ceph-mon[29604]: from='osd.1 v2:172.21.15.143:6805/507789371' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:27:20.260 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:20 smithi143 ceph-mon[38451]: from='osd.1 v2:172.21.15.143:6805/507789371' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T07:27:20.260 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:20 smithi143 ceph-mon[38451]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T07:27:20.260 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:20 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:20.260 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:20 smithi143 ceph-mon[38451]: from='osd.1 v2:172.21.15.143:6805/507789371' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:27:20.260 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:27:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-1[48887]: 2024-09-11T07:27:20.038+0000 7f8506ac3640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T07:27:20.261 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:27:20 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-1[48887]: 2024-09-11T07:27:20.042+0000 7f85000d5640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:27:20.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:20 smithi164 ceph-mon[33167]: from='osd.1 v2:172.21.15.143:6805/507789371' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T07:27:20.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:20 smithi164 ceph-mon[33167]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T07:27:20.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:20 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:20.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:20 smithi164 ceph-mon[33167]: from='osd.1 v2:172.21.15.143:6805/507789371' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:27:21.274 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:21 smithi143 ceph-mon[29604]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:21.274 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:21 smithi143 ceph-mon[29604]: from='osd.1 v2:172.21.15.143:6805/507789371' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-11T07:27:21.275 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:21 smithi143 ceph-mon[29604]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T07:27:21.275 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:21 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:21.275 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:21 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:21.275 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:21 smithi143 ceph-mon[38451]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:21.275 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:21 smithi143 ceph-mon[38451]: from='osd.1 v2:172.21.15.143:6805/507789371' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-11T07:27:21.275 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:21 smithi143 ceph-mon[38451]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T07:27:21.275 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:21 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:21.275 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:21 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:21.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:21 smithi164 ceph-mon[33167]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:21.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:21 smithi164 ceph-mon[33167]: from='osd.1 v2:172.21.15.143:6805/507789371' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-11T07:27:21.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:21 smithi164 ceph-mon[33167]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T07:27:21.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:21 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:21.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:21 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:22.298 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:22 smithi143 ceph-mon[38451]: purged_snaps scrub starts 2024-09-11T07:27:22.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:22 smithi143 ceph-mon[38451]: purged_snaps scrub ok 2024-09-11T07:27:22.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:22 smithi143 ceph-mon[38451]: osd.1 v2:172.21.15.143:6805/507789371 boot 2024-09-11T07:27:22.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:22 smithi143 ceph-mon[38451]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T07:27:22.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:22 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:22.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:22 smithi143 ceph-mon[29604]: purged_snaps scrub starts 2024-09-11T07:27:22.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:22 smithi143 ceph-mon[29604]: purged_snaps scrub ok 2024-09-11T07:27:22.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:22 smithi143 ceph-mon[29604]: osd.1 v2:172.21.15.143:6805/507789371 boot 2024-09-11T07:27:22.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:22 smithi143 ceph-mon[29604]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T07:27:22.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:22 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:22.394 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-11T07:27:22.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:22 smithi164 ceph-mon[33167]: purged_snaps scrub starts 2024-09-11T07:27:22.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:22 smithi164 ceph-mon[33167]: purged_snaps scrub ok 2024-09-11T07:27:22.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:22 smithi164 ceph-mon[33167]: osd.1 v2:172.21.15.143:6805/507789371 boot 2024-09-11T07:27:22.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:22 smithi164 ceph-mon[33167]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T07:27:22.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:22 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:22.480 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-11T07:27:22.494 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-09-11T07:27:22.495 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-09-11T07:27:22.495 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125204 s, 837 MB/s 2024-09-11T07:27:22.495 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-09-11T07:27:23.088 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:23 smithi143 ceph-mon[29604]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:23.088 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:23 smithi143 ceph-mon[29604]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T07:27:23.089 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:23 smithi143 ceph-mon[38451]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:23.089 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:23 smithi143 ceph-mon[38451]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T07:27:23.112 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_2 2024-09-11T07:27:23.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:23 smithi164 ceph-mon[33167]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:27:23.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:23 smithi164 ceph-mon[33167]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T07:27:23.671 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[29604]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[29604]: Detected new or changed devices on smithi143 2024-09-11T07:27:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[29604]: Adjusting osd_memory_target on smithi143 to 8032M 2024-09-11T07:27:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:27:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:24.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[38451]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:24.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[38451]: Detected new or changed devices on smithi143 2024-09-11T07:27:24.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:24.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:24.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:24.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:24.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[38451]: Adjusting osd_memory_target on smithi143 to 8032M 2024-09-11T07:27:24.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:24.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:24.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:27:24.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:24.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:24 smithi164 ceph-mon[33167]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:24.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:24 smithi164 ceph-mon[33167]: Detected new or changed devices on smithi143 2024-09-11T07:27:24.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:24.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:24.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:24.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:24.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:24 smithi164 ceph-mon[33167]: Adjusting osd_memory_target on smithi143 to 8032M 2024-09-11T07:27:24.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:24.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:24.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:27:24.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:25.943 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:25 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:27:25.943 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:25 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:27:25.943 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:25 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:25.943 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:25 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:27:25.944 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:25 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:27:25.944 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:25 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:25.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:25 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:27:25.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:25 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:27:25.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:25 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:26.904 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:26 smithi143 ceph-mon[29604]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:27:26.904 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:26 smithi143 ceph-mon[29604]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:26.904 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:26 smithi143 ceph-mon[38451]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:27:26.904 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:26 smithi143 ceph-mon[38451]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:26.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:26 smithi164 ceph-mon[33167]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:27:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:26 smithi164 ceph-mon[33167]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:27.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:27 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2029362880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cad9b00-6cc8-4cde-abe0-64f80bccb80b"}]: dispatch 2024-09-11T07:27:27.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:27 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2029362880' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cad9b00-6cc8-4cde-abe0-64f80bccb80b"}]': finished 2024-09-11T07:27:27.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:27 smithi164 ceph-mon[33167]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T07:27:27.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:27 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:28.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:27 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2029362880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cad9b00-6cc8-4cde-abe0-64f80bccb80b"}]: dispatch 2024-09-11T07:27:28.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:27 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2029362880' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cad9b00-6cc8-4cde-abe0-64f80bccb80b"}]': finished 2024-09-11T07:27:28.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:27 smithi143 ceph-mon[38451]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T07:27:28.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:27 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:28.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:27 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2029362880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cad9b00-6cc8-4cde-abe0-64f80bccb80b"}]: dispatch 2024-09-11T07:27:28.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:27 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2029362880' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cad9b00-6cc8-4cde-abe0-64f80bccb80b"}]': finished 2024-09-11T07:27:28.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:27 smithi143 ceph-mon[29604]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T07:27:28.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:27 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:28.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:28 smithi164 ceph-mon[33167]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:28.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:28 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/276618030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:27:29.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:28 smithi143 ceph-mon[38451]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:29.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:28 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/276618030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:27:29.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:28 smithi143 ceph-mon[29604]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:29.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:28 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/276618030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:27:30.870 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:30 smithi143 ceph-mon[29604]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:30.871 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:30 smithi143 ceph-mon[38451]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:30.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:30 smithi164 ceph-mon[33167]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:32.885 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:32 smithi143 ceph-mon[29604]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:32.885 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:32 smithi143 ceph-mon[38451]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:32.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:32 smithi164 ceph-mon[33167]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:33.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:33 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T07:27:33.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:33 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:33.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:33 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T07:27:33.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:33 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:33.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:33 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T07:27:33.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:33 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:34.771 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:34 smithi143 ceph-mon[29604]: Deploying daemon osd.2 on smithi143 2024-09-11T07:27:34.771 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:34 smithi143 ceph-mon[29604]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:34.771 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:34 smithi143 ceph-mon[38451]: Deploying daemon osd.2 on smithi143 2024-09-11T07:27:34.771 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:34 smithi143 ceph-mon[38451]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:34.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:34 smithi164 ceph-mon[33167]: Deploying daemon osd.2 on smithi143 2024-09-11T07:27:34.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:34 smithi164 ceph-mon[33167]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:36.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:36 smithi164 ceph-mon[33167]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:36.974 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:36 smithi143 ceph-mon[29604]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:36.974 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:36 smithi143 ceph-mon[38451]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:38.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:38 smithi143 ceph-mon[29604]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:38.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:38 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:27:38.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:38 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:38.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:38 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:38.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:38 smithi143 ceph-mon[38451]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:38.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:38 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:27:38.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:38 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:38.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:38 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:38.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:38 smithi164 ceph-mon[33167]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:38.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:38 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:27:38.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:38 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:38.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:38 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:40.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:40 smithi164 ceph-mon[33167]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:41.028 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:40 smithi143 ceph-mon[29604]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:41.028 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:40 smithi143 ceph-mon[38451]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:41.310 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 2 on host 'smithi143' 2024-09-11T07:27:41.922 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:41 smithi143 ceph-mon[38451]: from='osd.2 v2:172.21.15.143:6809/2660767941' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:27:41.922 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:41 smithi143 ceph-mon[38451]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:27:41.922 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:41 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:41.922 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:41 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:41.922 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:41 smithi143 ceph-mon[29604]: from='osd.2 v2:172.21.15.143:6809/2660767941' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:27:41.922 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:41 smithi143 ceph-mon[29604]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:27:41.923 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:41 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:41.923 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:41 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:41.946 DEBUG:teuthology.orchestra.run.smithi143:osd.2> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T07:27:41.949 INFO:tasks.cephadm:Deploying osd.3 on smithi143 with /dev/vg_nvme/lv_1... 2024-09-11T07:27:41.949 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-11T07:27:41.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:41 smithi164 ceph-mon[33167]: from='osd.2 v2:172.21.15.143:6809/2660767941' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:27:41.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:41 smithi164 ceph-mon[33167]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:27:41.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:41 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:41.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:41 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:42.249 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:42.945 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-mon[29604]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:42.945 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-mon[29604]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:27:42.945 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-mon[29604]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T07:27:42.945 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:42.945 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-mon[29604]: from='osd.2 v2:172.21.15.143:6809/2660767941' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:27:42.945 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-mon[29604]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:27:42.945 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:42.945 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:42.945 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-mon[38451]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:42.946 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-mon[38451]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:27:42.946 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-mon[38451]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T07:27:42.946 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:42.946 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-mon[38451]: from='osd.2 v2:172.21.15.143:6809/2660767941' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:27:42.946 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-mon[38451]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:27:42.946 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:42.946 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:42.946 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-2[55254]: 2024-09-11T07:27:42.699+0000 7fabb1dca640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T07:27:42.946 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:27:42 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-2[55254]: 2024-09-11T07:27:42.702+0000 7fabaec05640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:27:42.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:42 smithi164 ceph-mon[33167]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:42.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:42 smithi164 ceph-mon[33167]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:27:42.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:42 smithi164 ceph-mon[33167]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T07:27:42.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:42 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:42.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:42 smithi164 ceph-mon[33167]: from='osd.2 v2:172.21.15.143:6809/2660767941' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:27:42.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:42 smithi164 ceph-mon[33167]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:27:42.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:42 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:42.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:42 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:43.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:43 smithi164 ceph-mon[33167]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-11T07:27:43.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:43 smithi164 ceph-mon[33167]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T07:27:43.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:43 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:43.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:43 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:43.972 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-11T07:27:43.972 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:43 smithi143 ceph-mon[29604]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-11T07:27:43.972 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:43 smithi143 ceph-mon[29604]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T07:27:43.973 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:43 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:43.973 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:43 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:43.973 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:43 smithi143 ceph-mon[38451]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-11T07:27:43.973 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:43 smithi143 ceph-mon[38451]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T07:27:43.973 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:43 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:43.973 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:43 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:44.074 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-11T07:27:44.078 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-09-11T07:27:44.078 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-09-11T07:27:44.078 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125715 s, 834 MB/s 2024-09-11T07:27:44.078 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-09-11T07:27:44.796 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:44 smithi143 ceph-mon[29604]: purged_snaps scrub starts 2024-09-11T07:27:44.796 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:44 smithi143 ceph-mon[29604]: purged_snaps scrub ok 2024-09-11T07:27:44.796 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:44 smithi143 ceph-mon[29604]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:44.796 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:44 smithi143 ceph-mon[29604]: osd.2 v2:172.21.15.143:6809/2660767941 boot 2024-09-11T07:27:44.796 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:44 smithi143 ceph-mon[29604]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T07:27:44.796 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:44 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:44.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:44 smithi143 ceph-mon[38451]: purged_snaps scrub starts 2024-09-11T07:27:44.796 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:44 smithi143 ceph-mon[38451]: purged_snaps scrub ok 2024-09-11T07:27:44.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:44 smithi143 ceph-mon[38451]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:44.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:44 smithi143 ceph-mon[38451]: osd.2 v2:172.21.15.143:6809/2660767941 boot 2024-09-11T07:27:44.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:44 smithi143 ceph-mon[38451]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T07:27:44.797 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:44 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:44.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:44 smithi164 ceph-mon[33167]: purged_snaps scrub starts 2024-09-11T07:27:44.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:44 smithi164 ceph-mon[33167]: purged_snaps scrub ok 2024-09-11T07:27:44.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:44 smithi164 ceph-mon[33167]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:27:44.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:44 smithi164 ceph-mon[33167]: osd.2 v2:172.21.15.143:6809/2660767941 boot 2024-09-11T07:27:44.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:44 smithi164 ceph-mon[33167]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T07:27:44.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:44 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:45.264 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_1 2024-09-11T07:27:45.781 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:46.170 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:45 smithi143 ceph-mon[38451]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T07:27:46.170 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:45 smithi143 ceph-mon[29604]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T07:27:46.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:45 smithi164 ceph-mon[33167]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T07:27:46.992 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:46 smithi143 ceph-mon[29604]: pgmap v73: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:27:46.992 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:46 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T07:27:46.992 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:46 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T07:27:46.992 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:46 smithi143 ceph-mon[29604]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T07:27:46.992 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:46 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:27:46.992 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:46 smithi143 ceph-mon[38451]: pgmap v73: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:27:46.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:46 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T07:27:46.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:46 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T07:27:46.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:46 smithi143 ceph-mon[38451]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T07:27:46.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:46 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:27:47.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:46 smithi164 ceph-mon[33167]: pgmap v73: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:27:47.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:46 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T07:27:47.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:46 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T07:27:47.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:46 smithi164 ceph-mon[33167]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T07:27:47.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:46 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:27:47.574 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:27:47 smithi143 sudo[59128]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:27:47.575 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:27:47 smithi143 sudo[59128]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:27:47.575 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:27:47 smithi143 sudo[59128]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:27:47.575 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:27:47 smithi143 sudo[59128]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:27:47.831 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:27:47 smithi143 sudo[59163]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:27:47.831 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:27:47 smithi143 sudo[59163]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:27:47.831 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:27:47 smithi143 sudo[59163]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:27:47.831 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:27:47 smithi143 sudo[59163]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:27:48.118 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:27:48.118 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[29604]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T07:27:48.118 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.118 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.118 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:48.118 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:48.118 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:48.118 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.118 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:48.119 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:27:48.119 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.119 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:27:48.119 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:48.119 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:27:48.119 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.119 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.119 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:27:48.119 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[38451]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T07:27:48.119 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.119 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.120 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:48.120 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:48.120 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:48.120 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.120 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:48.120 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:27:48.120 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.120 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:27:48.120 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:48.120 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:27:48.120 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.120 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:47 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.120 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:27:47 smithi143 sudo[59224]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:27:48.121 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:27:47 smithi143 sudo[59224]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:27:48.121 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:27:47 smithi143 sudo[59224]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:27:48.121 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:27:47 smithi143 sudo[59224]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:27:48.121 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:27:48 smithi143 sudo[59261]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:27:48.377 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:27:48 smithi143 sudo[59261]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:27:48.377 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:27:48 smithi143 sudo[59261]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:27:48.377 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:27:48 smithi143 sudo[59261]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:27:48.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:47 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:27:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:47 smithi164 ceph-mon[33167]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T07:27:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:47 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:47 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:47 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:47 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:47 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:47 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:47 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:47 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:27:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:47 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:47 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:27:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:47 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:47 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:27:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:47 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:47 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:48.652 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:27:48 smithi143 sudo[59356]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:27:48.652 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:27:48 smithi143 sudo[59356]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:27:48.652 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:27:48 smithi143 sudo[59356]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:27:48.652 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:27:48 smithi143 sudo[59356]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:27:48.915 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:27:48 smithi143 sudo[59394]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:27:48.915 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:27:48 smithi143 sudo[59394]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:27:48.915 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:27:48 smithi143 sudo[59394]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:27:48.915 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:27:48 smithi143 sudo[59394]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:27:48.915 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:48 smithi143 sudo[59416]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:27:48.915 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:48 smithi143 sudo[59416]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:27:48.915 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:48 smithi143 sudo[59416]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:27:48.916 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:48 smithi143 sudo[59416]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:27:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:49 smithi143 sudo[59425]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:27:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:49 smithi143 sudo[59425]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:27:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:49 smithi143 sudo[59425]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:27:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:49 smithi143 sudo[59425]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:27:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[29604]: Detected new or changed devices on smithi143 2024-09-11T07:27:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[29604]: Adjusting osd_memory_target on smithi143 to 5354M 2024-09-11T07:27:49.167 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[29604]: pgmap v76: 1 pgs: 1 active+clean; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:27:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[29604]: osdmap e22: 3 total, 3 up, 3 in 2024-09-11T07:27:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[29604]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:27:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:27:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:27:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[29604]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T07:27:49.168 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[29604]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:27:49.424 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 ceph-mon[33167]: Detected new or changed devices on smithi143 2024-09-11T07:27:49.424 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 ceph-mon[33167]: Adjusting osd_memory_target on smithi143 to 5354M 2024-09-11T07:27:49.424 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 ceph-mon[33167]: pgmap v76: 1 pgs: 1 active+clean; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:27:49.424 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 ceph-mon[33167]: osdmap e22: 3 total, 3 up, 3 in 2024-09-11T07:27:49.424 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 ceph-mon[33167]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:27:49.424 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:27:49.424 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:27:49.424 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:49.425 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 ceph-mon[33167]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T07:27:49.425 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 ceph-mon[33167]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:27:49.425 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 sudo[35907]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:27:49.425 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 sudo[35907]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:27:49.425 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 sudo[35907]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:27:49.425 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 sudo[35907]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:27:49.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[38451]: Detected new or changed devices on smithi143 2024-09-11T07:27:49.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[38451]: Adjusting osd_memory_target on smithi143 to 5354M 2024-09-11T07:27:49.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[38451]: pgmap v76: 1 pgs: 1 active+clean; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:27:49.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[38451]: osdmap e22: 3 total, 3 up, 3 in 2024-09-11T07:27:49.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[38451]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:27:49.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:27:49.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:27:49.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:49.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[38451]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T07:27:49.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:49 smithi143 ceph-mon[38451]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:27:49.675 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 sudo[35910]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:27:49.675 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 sudo[35910]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:27:49.675 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 sudo[35910]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:27:49.675 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:49 smithi164 sudo[35910]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:27:49.974 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:49 smithi143 sudo[59478]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:27:49.974 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:49 smithi143 sudo[59478]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:27:49.974 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:49 smithi143 sudo[59478]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:27:49.974 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:49 smithi143 sudo[59478]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:27:50.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[29604]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:27:50.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:27:50.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:27:50.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:27:50.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[29604]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:27:50.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:27:50.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:27:50.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:27:50.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[29604]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:27:50.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:50.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:27:50.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:27:50.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:27:50.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 sudo[59520]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:27:50.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 sudo[59520]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:27:50.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 sudo[59520]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:27:50.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 sudo[59520]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:27:50.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[38451]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:27:50.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:27:50.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:27:50.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:27:50.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[38451]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:27:50.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:27:50.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:27:50.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:27:50.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[38451]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:27:50.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:50.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:27:50.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:27:50.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:50 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:27:50.591 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:50 smithi164 ceph-mon[33167]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:27:50.591 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:50 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:27:50.591 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:50 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:27:50.591 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:50 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:27:50.591 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:50 smithi164 ceph-mon[33167]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:27:50.591 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:50 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:27:50.591 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:50 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:27:50.591 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:50 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:27:50.592 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:50 smithi164 ceph-mon[33167]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:27:50.592 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:50 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:27:50.592 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:50 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:27:50.592 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:50 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:27:50.592 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:50 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:27:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:51 smithi143 ceph-mon[38451]: pgmap v78: 1 pgs: 1 active+clean; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:27:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:51 smithi143 ceph-mon[38451]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:27:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:51 smithi143 ceph-mon[38451]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:27:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:51 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1907587126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4a7c5e6-6afd-4bfc-8db9-ee6c4349061f"}]: dispatch 2024-09-11T07:27:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:51 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1907587126' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4a7c5e6-6afd-4bfc-8db9-ee6c4349061f"}]': finished 2024-09-11T07:27:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:51 smithi143 ceph-mon[38451]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T07:27:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:51 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:27:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:51 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3853064899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:27:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:51 smithi143 ceph-mon[29604]: pgmap v78: 1 pgs: 1 active+clean; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:27:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:51 smithi143 ceph-mon[29604]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:27:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:51 smithi143 ceph-mon[29604]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:27:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:51 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1907587126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4a7c5e6-6afd-4bfc-8db9-ee6c4349061f"}]: dispatch 2024-09-11T07:27:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:51 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1907587126' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4a7c5e6-6afd-4bfc-8db9-ee6c4349061f"}]': finished 2024-09-11T07:27:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:51 smithi143 ceph-mon[29604]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T07:27:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:51 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:27:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:51 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3853064899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:27:51.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:51 smithi164 ceph-mon[33167]: pgmap v78: 1 pgs: 1 active+clean; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:27:51.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:51 smithi164 ceph-mon[33167]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:27:51.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:51 smithi164 ceph-mon[33167]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:27:51.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:51 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1907587126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4a7c5e6-6afd-4bfc-8db9-ee6c4349061f"}]: dispatch 2024-09-11T07:27:51.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:51 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1907587126' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4a7c5e6-6afd-4bfc-8db9-ee6c4349061f"}]': finished 2024-09-11T07:27:51.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:51 smithi164 ceph-mon[33167]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T07:27:51.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:51 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:27:51.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:51 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3853064899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:27:53.479 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:53 smithi143 ceph-mon[29604]: pgmap v80: 1 pgs: 1 active+clean; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:27:53.479 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:53 smithi143 ceph-mon[38451]: pgmap v80: 1 pgs: 1 active+clean; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:27:53.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:53 smithi164 ceph-mon[33167]: pgmap v80: 1 pgs: 1 active+clean; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:27:55.512 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:55 smithi143 ceph-mon[29604]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 6.8 KiB/s rd, 271 KiB/s wr, 22 op/s 2024-09-11T07:27:55.512 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:55 smithi143 ceph-mon[38451]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 6.8 KiB/s rd, 271 KiB/s wr, 22 op/s 2024-09-11T07:27:55.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:55 smithi164 ceph-mon[33167]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 6.8 KiB/s rd, 271 KiB/s wr, 22 op/s 2024-09-11T07:27:56.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:56 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T07:27:56.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:56 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:56.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:56 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T07:27:56.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:56 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:56.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:56 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T07:27:56.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:56 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:57.505 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:57 smithi143 ceph-mon[29604]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.1 KiB/s rd, 203 KiB/s wr, 16 op/s 2024-09-11T07:27:57.505 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:57 smithi143 ceph-mon[29604]: Deploying daemon osd.3 on smithi143 2024-09-11T07:27:57.505 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:57 smithi143 ceph-mon[38451]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.1 KiB/s rd, 203 KiB/s wr, 16 op/s 2024-09-11T07:27:57.506 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:57 smithi143 ceph-mon[38451]: Deploying daemon osd.3 on smithi143 2024-09-11T07:27:57.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:57 smithi164 ceph-mon[33167]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.1 KiB/s rd, 203 KiB/s wr, 16 op/s 2024-09-11T07:27:57.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:57 smithi164 ceph-mon[33167]: Deploying daemon osd.3 on smithi143 2024-09-11T07:27:59.499 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:27:59 smithi143 ceph-mon[29604]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.2 KiB/s rd, 169 KiB/s wr, 13 op/s 2024-09-11T07:27:59.644 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:27:59 smithi143 ceph-mon[38451]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.2 KiB/s rd, 169 KiB/s wr, 13 op/s 2024-09-11T07:27:59.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:27:59 smithi164 ceph-mon[33167]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.2 KiB/s rd, 169 KiB/s wr, 13 op/s 2024-09-11T07:28:01.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:01 smithi143 ceph-mon[29604]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.1 KiB/s rd, 163 KiB/s wr, 13 op/s 2024-09-11T07:28:01.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:01 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:01.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:01 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:01.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:01 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:01.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:01 smithi143 ceph-mon[38451]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.1 KiB/s rd, 163 KiB/s wr, 13 op/s 2024-09-11T07:28:01.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:01 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:01.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:01 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:01.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:01 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:01.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:01 smithi164 ceph-mon[33167]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.1 KiB/s rd, 163 KiB/s wr, 13 op/s 2024-09-11T07:28:01.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:01 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:01.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:01 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:01.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:01 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:03.405 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:03 smithi143 ceph-mon[29604]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.6 KiB/s rd, 144 KiB/s wr, 11 op/s 2024-09-11T07:28:03.405 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:03 smithi143 ceph-mon[38451]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.6 KiB/s rd, 144 KiB/s wr, 11 op/s 2024-09-11T07:28:03.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:03 smithi164 ceph-mon[33167]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.6 KiB/s rd, 144 KiB/s wr, 11 op/s 2024-09-11T07:28:04.514 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 3 on host 'smithi143' 2024-09-11T07:28:05.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[29604]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.4 KiB/s rd, 136 KiB/s wr, 11 op/s 2024-09-11T07:28:05.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:05.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:05.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:05.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:05.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:05.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[29604]: from='osd.3 v2:172.21.15.143:6813/2376878046' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:28:05.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[29604]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:28:05.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:05.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:05.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:05.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[38451]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.4 KiB/s rd, 136 KiB/s wr, 11 op/s 2024-09-11T07:28:05.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:05.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:05.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:05.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:05.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:05.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[38451]: from='osd.3 v2:172.21.15.143:6813/2376878046' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:28:05.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[38451]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:28:05.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:05.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:05.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:04 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:05.106 DEBUG:teuthology.orchestra.run.smithi143:osd.3> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T07:28:05.109 INFO:tasks.cephadm:Deploying osd.4 on smithi164 with /dev/vg_nvme/lv_4... 2024-09-11T07:28:05.109 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-11T07:28:05.137 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:04 smithi164 ceph-mon[33167]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.4 KiB/s rd, 136 KiB/s wr, 11 op/s 2024-09-11T07:28:05.137 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:04 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:05.137 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:04 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:05.137 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:04 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:05.137 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:04 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:05.138 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:04 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:05.138 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:04 smithi164 ceph-mon[33167]: from='osd.3 v2:172.21.15.143:6813/2376878046' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:28:05.138 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:04 smithi164 ceph-mon[33167]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:28:05.138 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:04 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:05.138 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:04 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:05.138 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:04 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:05.364 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:28:06.272 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:06 smithi143 ceph-mon[29604]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T07:28:06.273 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:06 smithi143 ceph-mon[29604]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T07:28:06.273 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:06 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:28:06.273 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:06 smithi143 ceph-mon[29604]: from='osd.3 v2:172.21.15.143:6813/2376878046' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:28:06.273 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:06 smithi143 ceph-mon[29604]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:28:06.273 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:06 smithi143 ceph-mon[38451]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T07:28:06.273 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:06 smithi143 ceph-mon[38451]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T07:28:06.273 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:06 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:28:06.273 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:06 smithi143 ceph-mon[38451]: from='osd.3 v2:172.21.15.143:6813/2376878046' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:28:06.273 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:06 smithi143 ceph-mon[38451]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:28:06.274 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:28:06 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-3[61301]: 2024-09-11T07:28:06.250+0000 7fe05766d640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T07:28:06.274 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:28:06 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-3[61301]: 2024-09-11T07:28:06.254+0000 7fe0544a8640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:28:06.370 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:06 smithi164 ceph-mon[33167]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T07:28:06.371 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:06 smithi164 ceph-mon[33167]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T07:28:06.371 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:06 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:28:06.371 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:06 smithi164 ceph-mon[33167]: from='osd.3 v2:172.21.15.143:6813/2376878046' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:28:06.371 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:06 smithi164 ceph-mon[33167]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-11T07:28:07.005 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-11T07:28:07.113 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-11T07:28:07.126 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-09-11T07:28:07.126 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-09-11T07:28:07.126 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113293 s, 926 MB/s 2024-09-11T07:28:07.126 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-09-11T07:28:07.502 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:07 smithi143 ceph-mon[29604]: purged_snaps scrub starts 2024-09-11T07:28:07.503 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:07 smithi143 ceph-mon[29604]: purged_snaps scrub ok 2024-09-11T07:28:07.503 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:07 smithi143 ceph-mon[29604]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:28:07.503 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:07 smithi143 ceph-mon[29604]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-11T07:28:07.503 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:07 smithi143 ceph-mon[29604]: osdmap e25: 4 total, 3 up, 4 in 2024-09-11T07:28:07.503 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:07 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:28:07.503 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:07 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:28:07.503 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:07 smithi143 ceph-mon[38451]: purged_snaps scrub starts 2024-09-11T07:28:07.503 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:07 smithi143 ceph-mon[38451]: purged_snaps scrub ok 2024-09-11T07:28:07.503 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:07 smithi143 ceph-mon[38451]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:28:07.503 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:07 smithi143 ceph-mon[38451]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-11T07:28:07.503 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:07 smithi143 ceph-mon[38451]: osdmap e25: 4 total, 3 up, 4 in 2024-09-11T07:28:07.504 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:07 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:28:07.504 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:07 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:28:07.651 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:07 smithi164 ceph-mon[33167]: purged_snaps scrub starts 2024-09-11T07:28:07.651 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:07 smithi164 ceph-mon[33167]: purged_snaps scrub ok 2024-09-11T07:28:07.652 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:07 smithi164 ceph-mon[33167]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:28:07.652 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:07 smithi164 ceph-mon[33167]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-11T07:28:07.652 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:07 smithi164 ceph-mon[33167]: osdmap e25: 4 total, 3 up, 4 in 2024-09-11T07:28:07.652 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:07 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:28:07.652 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:07 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:28:07.676 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_4 2024-09-11T07:28:07.958 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:28:08.405 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:28:08.405 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: osd.3 v2:172.21.15.143:6813/2376878046 boot 2024-09-11T07:28:08.405 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T07:28:08.405 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:28:08.405 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:28:08.405 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: Detected new or changed devices on smithi143 2024-09-11T07:28:08.405 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:08.405 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:08.406 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:08.406 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:08.406 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:08.406 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:08.406 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: Adjusting osd_memory_target on smithi143 to 4016M 2024-09-11T07:28:08.406 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:08.406 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:08.406 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:08.406 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:08 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:28:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: osd.3 v2:172.21.15.143:6813/2376878046 boot 2024-09-11T07:28:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T07:28:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:28:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:28:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: Detected new or changed devices on smithi143 2024-09-11T07:28:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:08.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:08.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:08.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:08.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: Adjusting osd_memory_target on smithi143 to 4016M 2024-09-11T07:28:08.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:08.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:08.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:08.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:08.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:28:08.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: osd.3 v2:172.21.15.143:6813/2376878046 boot 2024-09-11T07:28:08.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T07:28:08.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:28:08.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:28:08.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: Detected new or changed devices on smithi143 2024-09-11T07:28:08.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:08.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:08.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:08.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:08.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:08.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:08.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: Adjusting osd_memory_target on smithi143 to 4016M 2024-09-11T07:28:08.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:08.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:08.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:08.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:08 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:09.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:09 smithi164 ceph-mon[33167]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T07:28:09.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:09 smithi143 ceph-mon[29604]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T07:28:09.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:09 smithi143 ceph-mon[38451]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T07:28:10.413 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:10 smithi164 ceph-mon[33167]: from='client.24184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:28:10.413 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:10 smithi164 ceph-mon[33167]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:10.413 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:10 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:28:10.413 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:10 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:28:10.413 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:10 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:10.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:10 smithi143 ceph-mon[29604]: from='client.24184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:28:10.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:10 smithi143 ceph-mon[29604]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:10.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:10 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:28:10.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:10 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:28:10.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:10 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:10.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:10 smithi143 ceph-mon[38451]: from='client.24184 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:28:10.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:10 smithi143 ceph-mon[38451]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:10.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:10 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:28:10.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:10 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:28:10.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:10 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:12.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:12 smithi164 ceph-mon[33167]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:12.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:12 smithi164 ceph-mon[33167]: from='client.? 172.21.15.164:0/1005482554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61aa7852-9e2a-4498-9f68-b248012431a3"}]: dispatch 2024-09-11T07:28:12.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:12 smithi164 ceph-mon[33167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61aa7852-9e2a-4498-9f68-b248012431a3"}]: dispatch 2024-09-11T07:28:12.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:12 smithi164 ceph-mon[33167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61aa7852-9e2a-4498-9f68-b248012431a3"}]': finished 2024-09-11T07:28:12.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:12 smithi164 ceph-mon[33167]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T07:28:12.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:12 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:28:12.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:12 smithi164 ceph-mon[33167]: from='client.? 172.21.15.164:0/4217485882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:28:13.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:12 smithi143 ceph-mon[38451]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:13.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:12 smithi143 ceph-mon[38451]: from='client.? 172.21.15.164:0/1005482554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61aa7852-9e2a-4498-9f68-b248012431a3"}]: dispatch 2024-09-11T07:28:13.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:12 smithi143 ceph-mon[38451]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61aa7852-9e2a-4498-9f68-b248012431a3"}]: dispatch 2024-09-11T07:28:13.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:12 smithi143 ceph-mon[38451]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61aa7852-9e2a-4498-9f68-b248012431a3"}]': finished 2024-09-11T07:28:13.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:12 smithi143 ceph-mon[38451]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T07:28:13.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:12 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:28:13.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:12 smithi143 ceph-mon[38451]: from='client.? 172.21.15.164:0/4217485882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:28:13.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:12 smithi143 ceph-mon[29604]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:13.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:12 smithi143 ceph-mon[29604]: from='client.? 172.21.15.164:0/1005482554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61aa7852-9e2a-4498-9f68-b248012431a3"}]: dispatch 2024-09-11T07:28:13.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:12 smithi143 ceph-mon[29604]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61aa7852-9e2a-4498-9f68-b248012431a3"}]: dispatch 2024-09-11T07:28:13.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:12 smithi143 ceph-mon[29604]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61aa7852-9e2a-4498-9f68-b248012431a3"}]': finished 2024-09-11T07:28:13.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:12 smithi143 ceph-mon[29604]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T07:28:13.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:12 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:28:13.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:12 smithi143 ceph-mon[29604]: from='client.? 172.21.15.164:0/4217485882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:28:14.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:14 smithi164 ceph-mon[33167]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:15.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:14 smithi143 ceph-mon[29604]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:15.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:14 smithi143 ceph-mon[38451]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:16.929 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:16 smithi164 ceph-mon[33167]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:17.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:16 smithi143 ceph-mon[29604]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:17.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:16 smithi143 ceph-mon[38451]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:18.948 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:18 smithi164 ceph-mon[33167]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:18.949 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T07:28:18.949 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:18.949 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:18 smithi164 ceph-mon[33167]: Deploying daemon osd.4 on smithi164 2024-09-11T07:28:19.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:18 smithi143 ceph-mon[38451]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:19.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T07:28:19.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:19.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:18 smithi143 ceph-mon[38451]: Deploying daemon osd.4 on smithi164 2024-09-11T07:28:19.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:18 smithi143 ceph-mon[29604]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:19.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T07:28:19.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:19.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:18 smithi143 ceph-mon[29604]: Deploying daemon osd.4 on smithi164 2024-09-11T07:28:20.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:20 smithi164 ceph-mon[33167]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:21.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:20 smithi143 ceph-mon[29604]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:21.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:20 smithi143 ceph-mon[38451]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:22.990 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:22 smithi164 ceph-mon[33167]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:22.990 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:22 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:22.990 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:22 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:22.990 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:22 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:23.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:22 smithi143 ceph-mon[29604]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:23.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:22 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:23.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:22 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:23.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:22 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:23.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:22 smithi143 ceph-mon[38451]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:23.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:22 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:23.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:22 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:23.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:22 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:24.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:24 smithi164 ceph-mon[33167]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:24.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:24.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:24 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:25.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:24 smithi143 ceph-mon[29604]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:25.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:25.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:24 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:25.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:24 smithi143 ceph-mon[38451]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:25.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:25.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:24 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:25.837 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:25 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:25.838 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:25 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:25.838 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:25 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:25.838 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:25 smithi164 ceph-mon[33167]: from='osd.4 v2:172.21.15.164:6800/2286891940' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:28:25.838 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:25 smithi164 ceph-mon[33167]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:28:25.874 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 4 on host 'smithi164' 2024-09-11T07:28:26.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:25 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:26.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:25 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:26.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:25 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:26.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:25 smithi143 ceph-mon[29604]: from='osd.4 v2:172.21.15.164:6800/2286891940' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:28:26.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:25 smithi143 ceph-mon[29604]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:28:26.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:25 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:26.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:25 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:26.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:25 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:26.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:25 smithi143 ceph-mon[38451]: from='osd.4 v2:172.21.15.164:6800/2286891940' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:28:26.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:25 smithi143 ceph-mon[38451]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:28:26.476 DEBUG:teuthology.orchestra.run.smithi164:osd.4> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T07:28:26.478 INFO:tasks.cephadm:Deploying osd.5 on smithi164 with /dev/vg_nvme/lv_3... 2024-09-11T07:28:26.478 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-11T07:28:26.993 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:26 smithi164 ceph-mon[33167]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:26.993 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:26.993 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:26.993 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:26.994 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:26 smithi164 ceph-mon[33167]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T07:28:26.994 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:26 smithi164 ceph-mon[33167]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T07:28:26.994 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:28:26.994 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:26 smithi164 ceph-mon[33167]: from='osd.4 v2:172.21.15.164:6800/2286891940' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:28:26.994 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:26 smithi164 ceph-mon[33167]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:28:27.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[38451]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:27.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:27.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:27.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:27.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[38451]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T07:28:27.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[38451]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T07:28:27.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:28:27.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[38451]: from='osd.4 v2:172.21.15.164:6800/2286891940' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:28:27.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[38451]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:28:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[29604]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[29604]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T07:28:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[29604]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T07:28:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:28:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[29604]: from='osd.4 v2:172.21.15.164:6800/2286891940' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:28:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:26 smithi143 ceph-mon[29604]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:28:27.304 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:28:27 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4[38495]: 2024-09-11T07:28:27.100+0000 7fe629497640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T07:28:27.305 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:28:27 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4[38495]: 2024-09-11T07:28:27.104+0000 7fe6262d2640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:28:27.657 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:28:28.160 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:28 smithi164 ceph-mon[33167]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-11T07:28:28.160 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:28 smithi164 ceph-mon[33167]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T07:28:28.160 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:28 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:28:28.160 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:28 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:28:28.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:28 smithi143 ceph-mon[29604]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-11T07:28:28.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:28 smithi143 ceph-mon[29604]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T07:28:28.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:28 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:28:28.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:28 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:28:28.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:28 smithi143 ceph-mon[38451]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-11T07:28:28.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:28 smithi143 ceph-mon[38451]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T07:28:28.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:28 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:28:28.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:28 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:28:29.383 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:29 smithi164 ceph-mon[33167]: purged_snaps scrub starts 2024-09-11T07:28:29.383 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:29 smithi164 ceph-mon[33167]: purged_snaps scrub ok 2024-09-11T07:28:29.383 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:29 smithi164 ceph-mon[33167]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:29.383 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:29 smithi164 ceph-mon[33167]: osd.4 v2:172.21.15.164:6800/2286891940 boot 2024-09-11T07:28:29.383 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:29 smithi164 ceph-mon[33167]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T07:28:29.383 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:29 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:28:29.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:29 smithi143 ceph-mon[29604]: purged_snaps scrub starts 2024-09-11T07:28:29.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:29 smithi143 ceph-mon[29604]: purged_snaps scrub ok 2024-09-11T07:28:29.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:29 smithi143 ceph-mon[29604]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:29.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:29 smithi143 ceph-mon[29604]: osd.4 v2:172.21.15.164:6800/2286891940 boot 2024-09-11T07:28:29.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:29 smithi143 ceph-mon[29604]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T07:28:29.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:29 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:28:29.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:29 smithi143 ceph-mon[38451]: purged_snaps scrub starts 2024-09-11T07:28:29.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:29 smithi143 ceph-mon[38451]: purged_snaps scrub ok 2024-09-11T07:28:29.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:29 smithi143 ceph-mon[38451]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:28:29.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:29 smithi143 ceph-mon[38451]: osd.4 v2:172.21.15.164:6800/2286891940 boot 2024-09-11T07:28:29.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:29 smithi143 ceph-mon[38451]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T07:28:29.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:29 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:28:29.853 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-11T07:28:29.935 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-11T07:28:29.949 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-09-11T07:28:29.949 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-09-11T07:28:29.949 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125922 s, 833 MB/s 2024-09-11T07:28:29.949 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-09-11T07:28:30.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:30 smithi164 ceph-mon[33167]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T07:28:30.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:30 smithi164 ceph-mon[33167]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T07:28:30.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:30 smithi143 ceph-mon[29604]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T07:28:30.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:30 smithi143 ceph-mon[29604]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T07:28:30.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:30 smithi143 ceph-mon[38451]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T07:28:30.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:30 smithi143 ceph-mon[38451]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T07:28:30.891 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_3 2024-09-11T07:28:31.142 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:28:31.377 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:31 smithi164 ceph-mon[33167]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:31.377 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:31 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:31.377 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:31 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:31.377 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:31 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:31.377 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:31 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:31.377 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:31 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:31.377 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:31 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:31.378 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:31 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:31.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:31 smithi143 ceph-mon[29604]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:31.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:31 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:31.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:31 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:31.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:31 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:31.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:31 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:31.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:31 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:31.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:31 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:31.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:31 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:31.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:31 smithi143 ceph-mon[38451]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:31.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:31 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:31.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:31 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:31.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:31 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:31.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:31 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:31.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:31 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:31.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:31 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:31.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:31 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:32.329 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:32 smithi164 ceph-mon[33167]: Detected new or changed devices on smithi164 2024-09-11T07:28:32.329 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:32 smithi164 ceph-mon[33167]: Adjusting osd_memory_target on smithi164 to 17088M 2024-09-11T07:28:32.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:32 smithi143 ceph-mon[29604]: Detected new or changed devices on smithi164 2024-09-11T07:28:32.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:32 smithi143 ceph-mon[29604]: Adjusting osd_memory_target on smithi164 to 17088M 2024-09-11T07:28:32.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:32 smithi143 ceph-mon[38451]: Detected new or changed devices on smithi164 2024-09-11T07:28:32.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:32 smithi143 ceph-mon[38451]: Adjusting osd_memory_target on smithi164 to 17088M 2024-09-11T07:28:33.406 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:33 smithi164 ceph-mon[33167]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:33.406 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:33 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:28:33.406 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:33 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:28:33.406 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:33 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:33.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:33 smithi143 ceph-mon[29604]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:33.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:33 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:28:33.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:33 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:28:33.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:33 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:33.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:33 smithi143 ceph-mon[38451]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:33.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:33 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:28:33.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:33 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:28:33.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:33 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:34.229 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:34 smithi164 ceph-mon[33167]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:28:34.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:34 smithi143 ceph-mon[29604]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:28:34.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:34 smithi143 ceph-mon[38451]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:28:35.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:35 smithi164 ceph-mon[33167]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:28:35.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:35 smithi164 ceph-mon[33167]: from='client.? 172.21.15.164:0/2682094396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd3428b7-b0c6-474e-b462-d964c1b91327"}]: dispatch 2024-09-11T07:28:35.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:35 smithi164 ceph-mon[33167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd3428b7-b0c6-474e-b462-d964c1b91327"}]: dispatch 2024-09-11T07:28:35.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:35 smithi164 ceph-mon[33167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd3428b7-b0c6-474e-b462-d964c1b91327"}]': finished 2024-09-11T07:28:35.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:35 smithi164 ceph-mon[33167]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T07:28:35.469 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:35 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:28:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:35 smithi143 ceph-mon[29604]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:28:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:35 smithi143 ceph-mon[29604]: from='client.? 172.21.15.164:0/2682094396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd3428b7-b0c6-474e-b462-d964c1b91327"}]: dispatch 2024-09-11T07:28:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:35 smithi143 ceph-mon[29604]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd3428b7-b0c6-474e-b462-d964c1b91327"}]: dispatch 2024-09-11T07:28:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:35 smithi143 ceph-mon[29604]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd3428b7-b0c6-474e-b462-d964c1b91327"}]': finished 2024-09-11T07:28:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:35 smithi143 ceph-mon[29604]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T07:28:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:35 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:28:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:35 smithi143 ceph-mon[38451]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:28:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:35 smithi143 ceph-mon[38451]: from='client.? 172.21.15.164:0/2682094396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd3428b7-b0c6-474e-b462-d964c1b91327"}]: dispatch 2024-09-11T07:28:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:35 smithi143 ceph-mon[38451]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd3428b7-b0c6-474e-b462-d964c1b91327"}]: dispatch 2024-09-11T07:28:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:35 smithi143 ceph-mon[38451]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd3428b7-b0c6-474e-b462-d964c1b91327"}]': finished 2024-09-11T07:28:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:35 smithi143 ceph-mon[38451]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T07:28:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:35 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:28:36.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:36 smithi164 ceph-mon[33167]: from='client.? 172.21.15.164:0/2534998839' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:28:36.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:36 smithi143 ceph-mon[29604]: from='client.? 172.21.15.164:0/2534998839' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:28:36.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:36 smithi143 ceph-mon[38451]: from='client.? 172.21.15.164:0/2534998839' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:28:37.450 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:37 smithi143 ceph-mon[29604]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-11T07:28:37.450 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:37 smithi143 ceph-mon[38451]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-11T07:28:37.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:37 smithi164 ceph-mon[33167]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-11T07:28:39.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:39 smithi143 ceph-mon[29604]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T07:28:39.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:39 smithi143 ceph-mon[38451]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T07:28:39.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:39 smithi164 ceph-mon[33167]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T07:28:41.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:41 smithi164 ceph-mon[33167]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T07:28:41.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:41 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T07:28:41.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:41 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:41.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:41 smithi143 ceph-mon[29604]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T07:28:41.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:41 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T07:28:41.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:41 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:41.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:41 smithi143 ceph-mon[38451]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T07:28:41.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:41 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T07:28:41.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:41 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:42.321 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:42 smithi164 ceph-mon[33167]: Deploying daemon osd.5 on smithi164 2024-09-11T07:28:42.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:42 smithi143 ceph-mon[29604]: Deploying daemon osd.5 on smithi164 2024-09-11T07:28:42.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:42 smithi143 ceph-mon[38451]: Deploying daemon osd.5 on smithi164 2024-09-11T07:28:43.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:43 smithi164 ceph-mon[33167]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:28:43.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:43 smithi143 ceph-mon[29604]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:28:43.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:43 smithi143 ceph-mon[38451]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:28:45.200 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:45 smithi164 ceph-mon[33167]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:45.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:45 smithi143 ceph-mon[29604]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:45.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:45 smithi143 ceph-mon[38451]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:46.419 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:46 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:46.419 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:46 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:46.419 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:46 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:46.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:46 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:46.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:46 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:46.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:46 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:46.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:46 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:46.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:46 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:46.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:46 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:47.432 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:47 smithi164 ceph-mon[33167]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:47.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:47 smithi143 ceph-mon[29604]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:47.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:47 smithi143 ceph-mon[38451]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:48.860 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 5 on host 'smithi164' 2024-09-11T07:28:49.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:49 smithi164 ceph-mon[33167]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:49 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:49 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:49 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:49 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:49 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:49 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:49 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:49 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:49 smithi164 ceph-mon[33167]: from='osd.5 v2:172.21.15.164:6804/4094877797' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:28:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:49 smithi164 ceph-mon[33167]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:28:49.536 DEBUG:teuthology.orchestra.run.smithi164:osd.5> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T07:28:49.539 INFO:tasks.cephadm:Deploying osd.6 on smithi164 with /dev/vg_nvme/lv_2... 2024-09-11T07:28:49.539 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-11T07:28:49.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[29604]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[29604]: from='osd.5 v2:172.21.15.164:6804/4094877797' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:28:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[29604]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:28:49.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[38451]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:49.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:49.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:49.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:49.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:49.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:49.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:49.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:49.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:49.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[38451]: from='osd.5 v2:172.21.15.164:6804/4094877797' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:28:49.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:49 smithi143 ceph-mon[38451]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:28:50.255 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:28:51.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:50 smithi164 ceph-mon[33167]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:51.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:50 smithi164 ceph-mon[33167]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T07:28:51.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:50 smithi164 ceph-mon[33167]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T07:28:51.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:50 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:28:51.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:50 smithi164 ceph-mon[33167]: from='osd.5 v2:172.21.15.164:6804/4094877797' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:28:51.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:50 smithi164 ceph-mon[33167]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:28:51.217 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:28:50 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:28:50.870+0000 7f5fb558f640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T07:28:51.217 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:28:50 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:28:50.875+0000 7f5faf3c4640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:28:51.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:50 smithi143 ceph-mon[29604]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:51.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:50 smithi143 ceph-mon[29604]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T07:28:51.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:50 smithi143 ceph-mon[29604]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T07:28:51.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:50 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:28:51.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:50 smithi143 ceph-mon[29604]: from='osd.5 v2:172.21.15.164:6804/4094877797' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:28:51.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:50 smithi143 ceph-mon[29604]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:28:51.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:50 smithi143 ceph-mon[38451]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:51.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:50 smithi143 ceph-mon[38451]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T07:28:51.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:50 smithi143 ceph-mon[38451]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T07:28:51.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:50 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:28:51.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:50 smithi143 ceph-mon[38451]: from='osd.5 v2:172.21.15.164:6804/4094877797' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:28:51.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:50 smithi143 ceph-mon[38451]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:28:51.866 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-11T07:28:51.949 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-11T07:28:51.963 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-09-11T07:28:51.963 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-09-11T07:28:51.963 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125163 s, 838 MB/s 2024-09-11T07:28:51.963 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-09-11T07:28:52.126 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:51 smithi164 ceph-mon[33167]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-11T07:28:52.127 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:51 smithi164 ceph-mon[33167]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T07:28:52.127 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:51 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:28:52.127 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:51 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:28:52.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:51 smithi143 ceph-mon[29604]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-11T07:28:52.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:51 smithi143 ceph-mon[29604]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T07:28:52.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:51 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:28:52.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:51 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:28:52.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:51 smithi143 ceph-mon[38451]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-11T07:28:52.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:51 smithi143 ceph-mon[38451]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T07:28:52.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:51 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:28:52.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:51 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:28:52.741 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_2 2024-09-11T07:28:52.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:52 smithi164 ceph-mon[33167]: purged_snaps scrub starts 2024-09-11T07:28:52.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:52 smithi164 ceph-mon[33167]: purged_snaps scrub ok 2024-09-11T07:28:52.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:52 smithi164 ceph-mon[33167]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:52.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:52 smithi164 ceph-mon[33167]: osd.5 v2:172.21.15.164:6804/4094877797 boot 2024-09-11T07:28:52.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:52 smithi164 ceph-mon[33167]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T07:28:52.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:52 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:28:52.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:52 smithi164 ceph-mon[33167]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T07:28:53.225 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:28:53.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:52 smithi143 ceph-mon[29604]: purged_snaps scrub starts 2024-09-11T07:28:53.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:52 smithi143 ceph-mon[29604]: purged_snaps scrub ok 2024-09-11T07:28:53.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:52 smithi143 ceph-mon[29604]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:53.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:52 smithi143 ceph-mon[29604]: osd.5 v2:172.21.15.164:6804/4094877797 boot 2024-09-11T07:28:53.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:52 smithi143 ceph-mon[29604]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T07:28:53.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:52 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:28:53.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:52 smithi143 ceph-mon[29604]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T07:28:53.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:52 smithi143 ceph-mon[38451]: purged_snaps scrub starts 2024-09-11T07:28:53.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:52 smithi143 ceph-mon[38451]: purged_snaps scrub ok 2024-09-11T07:28:53.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:52 smithi143 ceph-mon[38451]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:28:53.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:52 smithi143 ceph-mon[38451]: osd.5 v2:172.21.15.164:6804/4094877797 boot 2024-09-11T07:28:53.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:52 smithi143 ceph-mon[38451]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T07:28:53.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:52 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:28:53.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:52 smithi143 ceph-mon[38451]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T07:28:54.959 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:54 smithi164 ceph-mon[33167]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:28:54.959 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:54 smithi164 ceph-mon[33167]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T07:28:55.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:54 smithi143 ceph-mon[29604]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:28:55.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:54 smithi143 ceph-mon[29604]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T07:28:55.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:54 smithi143 ceph-mon[38451]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:28:55.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:54 smithi143 ceph-mon[38451]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T07:28:56.232 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:55 smithi164 ceph-mon[33167]: Detected new or changed devices on smithi164 2024-09-11T07:28:56.232 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:55 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:56.232 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:55 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:56.232 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:55 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:56.232 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:55 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:56.232 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:55 smithi164 ceph-mon[33167]: Adjusting osd_memory_target on smithi164 to 8544M 2024-09-11T07:28:56.232 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:55 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:56.232 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:55 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:56.232 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:55 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:56.232 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:55 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:56.232 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:55 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:28:56.232 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:55 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:28:56.233 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:55 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:56.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[29604]: Detected new or changed devices on smithi164 2024-09-11T07:28:56.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:56.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:56.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:56.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:56.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[29604]: Adjusting osd_memory_target on smithi164 to 8544M 2024-09-11T07:28:56.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:56.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:56.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:56.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:56.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:28:56.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:28:56.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:56.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[38451]: Detected new or changed devices on smithi164 2024-09-11T07:28:56.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:56.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:56.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:56.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:28:56.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[38451]: Adjusting osd_memory_target on smithi164 to 8544M 2024-09-11T07:28:56.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:56.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:56.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:56.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:28:56.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:28:56.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:28:56.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:55 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:57.269 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:57 smithi164 ceph-mon[33167]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:28:57.269 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:57 smithi164 ceph-mon[33167]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:28:57.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:57 smithi143 ceph-mon[29604]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:28:57.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:57 smithi143 ceph-mon[29604]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:28:57.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:57 smithi143 ceph-mon[38451]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:28:57.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:57 smithi143 ceph-mon[38451]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:28:58.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:58 smithi143 ceph-mon[29604]: from='client.? 172.21.15.164:0/3203261050' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abbb121a-3d68-4661-8087-f8630232871e"}]: dispatch 2024-09-11T07:28:58.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:58 smithi143 ceph-mon[29604]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abbb121a-3d68-4661-8087-f8630232871e"}]: dispatch 2024-09-11T07:28:58.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:58 smithi143 ceph-mon[29604]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abbb121a-3d68-4661-8087-f8630232871e"}]': finished 2024-09-11T07:28:58.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:58 smithi143 ceph-mon[29604]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T07:28:58.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:58 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:28:58.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:58 smithi143 ceph-mon[38451]: from='client.? 172.21.15.164:0/3203261050' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abbb121a-3d68-4661-8087-f8630232871e"}]: dispatch 2024-09-11T07:28:58.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:58 smithi143 ceph-mon[38451]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abbb121a-3d68-4661-8087-f8630232871e"}]: dispatch 2024-09-11T07:28:58.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:58 smithi143 ceph-mon[38451]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abbb121a-3d68-4661-8087-f8630232871e"}]': finished 2024-09-11T07:28:58.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:58 smithi143 ceph-mon[38451]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T07:28:58.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:58 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:28:58.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:58 smithi164 ceph-mon[33167]: from='client.? 172.21.15.164:0/3203261050' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abbb121a-3d68-4661-8087-f8630232871e"}]: dispatch 2024-09-11T07:28:58.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:58 smithi164 ceph-mon[33167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abbb121a-3d68-4661-8087-f8630232871e"}]: dispatch 2024-09-11T07:28:58.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:58 smithi164 ceph-mon[33167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abbb121a-3d68-4661-8087-f8630232871e"}]': finished 2024-09-11T07:28:58.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:58 smithi164 ceph-mon[33167]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T07:28:58.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:58 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:28:59.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:59 smithi143 ceph-mon[29604]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:28:59.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:28:59 smithi143 ceph-mon[29604]: from='client.? 172.21.15.164:0/3593083598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:28:59.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:59 smithi143 ceph-mon[38451]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:28:59.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:28:59 smithi143 ceph-mon[38451]: from='client.? 172.21.15.164:0/3593083598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:28:59.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:59 smithi164 ceph-mon[33167]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:28:59.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:28:59 smithi164 ceph-mon[33167]: from='client.? 172.21.15.164:0/3593083598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:29:01.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:01 smithi143 ceph-mon[29604]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:01.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:01 smithi143 ceph-mon[38451]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:01.317 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:01 smithi164 ceph-mon[33167]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:03.201 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:03 smithi164 ceph-mon[33167]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:03.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:03 smithi143 ceph-mon[29604]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:03.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:03 smithi143 ceph-mon[38451]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:04.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:04 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T07:29:04.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:04 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:04.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:04 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T07:29:04.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:04 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:04.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:04 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T07:29:04.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:04 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:05.251 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:05 smithi164 ceph-mon[33167]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:05.252 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:05 smithi164 ceph-mon[33167]: Deploying daemon osd.6 on smithi164 2024-09-11T07:29:05.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:05 smithi143 ceph-mon[29604]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:05.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:05 smithi143 ceph-mon[29604]: Deploying daemon osd.6 on smithi164 2024-09-11T07:29:05.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:05 smithi143 ceph-mon[38451]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:05.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:05 smithi143 ceph-mon[38451]: Deploying daemon osd.6 on smithi164 2024-09-11T07:29:07.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:07 smithi143 ceph-mon[29604]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:07.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:07 smithi143 ceph-mon[38451]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:07.329 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:07 smithi164 ceph-mon[33167]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:09.297 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:09 smithi164 ceph-mon[33167]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:09.297 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:09 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:09.297 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:09 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:09.297 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:09 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:09.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:09 smithi143 ceph-mon[29604]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:09.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:09 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:09.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:09 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:09.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:09 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:09.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:09 smithi143 ceph-mon[38451]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:09.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:09 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:09.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:09 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:09.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:09 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:11.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:11 smithi143 ceph-mon[29604]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:11.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:11 smithi143 ceph-mon[38451]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:11.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:11 smithi164 ceph-mon[33167]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:11.813 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 6 on host 'smithi164' 2024-09-11T07:29:12.413 DEBUG:teuthology.orchestra.run.smithi164:osd.6> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T07:29:12.416 INFO:tasks.cephadm:Deploying osd.7 on smithi164 with /dev/vg_nvme/lv_1... 2024-09-11T07:29:12.416 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-11T07:29:12.721 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:29:13.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:12 smithi164 ceph-mon[33167]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:13.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:12 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:13.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:12 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:13.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:12 smithi164 ceph-mon[33167]: from='osd.6 v2:172.21.15.164:6808/174920052' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:29:13.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:12 smithi164 ceph-mon[33167]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:29:13.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:12 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:13.058 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:12 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:13.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:12 smithi143 ceph-mon[29604]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:13.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:12 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:13.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:12 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:13.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:12 smithi143 ceph-mon[29604]: from='osd.6 v2:172.21.15.164:6808/174920052' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:29:13.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:12 smithi143 ceph-mon[29604]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:29:13.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:12 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:13.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:12 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:13.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:12 smithi143 ceph-mon[38451]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:13.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:12 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:13.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:12 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:13.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:12 smithi143 ceph-mon[38451]: from='osd.6 v2:172.21.15.164:6808/174920052' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:29:13.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:12 smithi143 ceph-mon[38451]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:29:13.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:12 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:13.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:12 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:14.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:13 smithi164 ceph-mon[33167]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T07:29:14.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:13 smithi164 ceph-mon[33167]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T07:29:14.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:13 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:29:14.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:13 smithi164 ceph-mon[33167]: from='osd.6 v2:172.21.15.164:6808/174920052' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:29:14.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:13 smithi164 ceph-mon[33167]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:29:14.217 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:29:13 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:29:13.821+0000 7f33c3ac6640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T07:29:14.217 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:29:13 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:29:13.829+0000 7f33bf0fe640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:29:14.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:13 smithi143 ceph-mon[29604]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T07:29:14.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:13 smithi143 ceph-mon[29604]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T07:29:14.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:13 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:29:14.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:13 smithi143 ceph-mon[29604]: from='osd.6 v2:172.21.15.164:6808/174920052' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:29:14.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:13 smithi143 ceph-mon[29604]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:29:14.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:13 smithi143 ceph-mon[38451]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T07:29:14.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:13 smithi143 ceph-mon[38451]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T07:29:14.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:13 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:29:14.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:13 smithi143 ceph-mon[38451]: from='osd.6 v2:172.21.15.164:6808/174920052' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:29:14.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:13 smithi143 ceph-mon[38451]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:29:14.863 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-11T07:29:14.951 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-11T07:29:14.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:14 smithi164 ceph-mon[33167]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:14.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:14 smithi164 ceph-mon[33167]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-11T07:29:14.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:14 smithi164 ceph-mon[33167]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T07:29:14.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:14 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:29:14.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:14 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:29:14.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:14 smithi164 ceph-mon[33167]: osd.6 v2:172.21.15.164:6808/174920052 boot 2024-09-11T07:29:14.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:14 smithi164 ceph-mon[33167]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T07:29:14.952 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:14 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:29:14.967 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-09-11T07:29:14.967 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-09-11T07:29:14.967 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0141377 s, 742 MB/s 2024-09-11T07:29:14.968 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-09-11T07:29:15.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:14 smithi143 ceph-mon[29604]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:15.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:14 smithi143 ceph-mon[29604]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-11T07:29:15.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:14 smithi143 ceph-mon[29604]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T07:29:15.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:14 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:29:15.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:14 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:29:15.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:14 smithi143 ceph-mon[29604]: osd.6 v2:172.21.15.164:6808/174920052 boot 2024-09-11T07:29:15.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:14 smithi143 ceph-mon[29604]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T07:29:15.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:14 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:29:15.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:14 smithi143 ceph-mon[38451]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:29:15.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:14 smithi143 ceph-mon[38451]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-11T07:29:15.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:14 smithi143 ceph-mon[38451]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T07:29:15.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:14 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:29:15.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:14 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:29:15.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:14 smithi143 ceph-mon[38451]: osd.6 v2:172.21.15.164:6808/174920052 boot 2024-09-11T07:29:15.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:14 smithi143 ceph-mon[38451]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T07:29:15.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:14 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:29:15.717 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_1 2024-09-11T07:29:15.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:15 smithi164 ceph-mon[33167]: purged_snaps scrub starts 2024-09-11T07:29:15.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:15 smithi164 ceph-mon[33167]: purged_snaps scrub ok 2024-09-11T07:29:15.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:15 smithi164 ceph-mon[33167]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T07:29:16.042 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:29:16.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:15 smithi143 ceph-mon[29604]: purged_snaps scrub starts 2024-09-11T07:29:16.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:15 smithi143 ceph-mon[29604]: purged_snaps scrub ok 2024-09-11T07:29:16.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:15 smithi143 ceph-mon[29604]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T07:29:16.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:15 smithi143 ceph-mon[38451]: purged_snaps scrub starts 2024-09-11T07:29:16.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:15 smithi143 ceph-mon[38451]: purged_snaps scrub ok 2024-09-11T07:29:16.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:15 smithi143 ceph-mon[38451]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T07:29:17.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:16 smithi164 ceph-mon[33167]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:17.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:16 smithi164 ceph-mon[33167]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T07:29:17.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:16 smithi164 ceph-mon[33167]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T07:29:17.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:16 smithi143 ceph-mon[29604]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:17.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:16 smithi143 ceph-mon[29604]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T07:29:17.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:16 smithi143 ceph-mon[29604]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T07:29:17.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:16 smithi143 ceph-mon[38451]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:17.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:16 smithi143 ceph-mon[38451]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T07:29:17.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:16 smithi143 ceph-mon[38451]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T07:29:18.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:18.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:29:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:29:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:29:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: Detected new or changed devices on smithi164 2024-09-11T07:29:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: Adjusting osd_memory_target on smithi164 to 5696M 2024-09-11T07:29:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:18.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:18.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:18.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:18.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:18.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:18 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:19.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:19.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:29:19.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:29:19.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:29:19.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:19.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: Detected new or changed devices on smithi164 2024-09-11T07:29:19.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:19.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:19.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:19.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:19.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:19.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: Adjusting osd_memory_target on smithi164 to 5696M 2024-09-11T07:29:19.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:19.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:19.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:19.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:19.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:19.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:19.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:19.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:19.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:19.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:29:19.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:29:19.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:29:19.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:19.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: Detected new or changed devices on smithi164 2024-09-11T07:29:19.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:19.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:19.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:19.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:19.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:19.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: Adjusting osd_memory_target on smithi164 to 5696M 2024-09-11T07:29:19.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:19.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:19.302 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:19.302 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:19.302 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:19.302 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:19.302 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:19.302 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:18 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:21.049 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:20 smithi164 ceph-mon[33167]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:21.049 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:20 smithi164 ceph-mon[33167]: from='client.? 172.21.15.164:0/3628207977' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3b1d8ce-1a3a-4da5-bcf2-46d1aeb38b95"}]: dispatch 2024-09-11T07:29:21.049 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:20 smithi164 ceph-mon[33167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3b1d8ce-1a3a-4da5-bcf2-46d1aeb38b95"}]: dispatch 2024-09-11T07:29:21.049 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:20 smithi164 ceph-mon[33167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3b1d8ce-1a3a-4da5-bcf2-46d1aeb38b95"}]': finished 2024-09-11T07:29:21.049 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:20 smithi164 ceph-mon[33167]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T07:29:21.049 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:20 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:29:21.049 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:20 smithi164 ceph-mon[33167]: from='client.? 172.21.15.164:0/3257798689' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:29:21.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:20 smithi143 ceph-mon[29604]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:21.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:20 smithi143 ceph-mon[29604]: from='client.? 172.21.15.164:0/3628207977' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3b1d8ce-1a3a-4da5-bcf2-46d1aeb38b95"}]: dispatch 2024-09-11T07:29:21.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:20 smithi143 ceph-mon[29604]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3b1d8ce-1a3a-4da5-bcf2-46d1aeb38b95"}]: dispatch 2024-09-11T07:29:21.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:20 smithi143 ceph-mon[29604]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3b1d8ce-1a3a-4da5-bcf2-46d1aeb38b95"}]': finished 2024-09-11T07:29:21.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:20 smithi143 ceph-mon[29604]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T07:29:21.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:20 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:29:21.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:20 smithi143 ceph-mon[29604]: from='client.? 172.21.15.164:0/3257798689' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:29:21.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:20 smithi143 ceph-mon[38451]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:21.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:20 smithi143 ceph-mon[38451]: from='client.? 172.21.15.164:0/3628207977' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3b1d8ce-1a3a-4da5-bcf2-46d1aeb38b95"}]: dispatch 2024-09-11T07:29:21.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:20 smithi143 ceph-mon[38451]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3b1d8ce-1a3a-4da5-bcf2-46d1aeb38b95"}]: dispatch 2024-09-11T07:29:21.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:20 smithi143 ceph-mon[38451]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3b1d8ce-1a3a-4da5-bcf2-46d1aeb38b95"}]': finished 2024-09-11T07:29:21.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:20 smithi143 ceph-mon[38451]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T07:29:21.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:20 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:29:21.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:20 smithi143 ceph-mon[38451]: from='client.? 172.21.15.164:0/3257798689' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:29:22.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:21 smithi164 ceph-mon[33167]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T07:29:22.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:21 smithi164 ceph-mon[33167]: Cluster is now healthy 2024-09-11T07:29:22.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:21 smithi143 ceph-mon[29604]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T07:29:22.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:21 smithi143 ceph-mon[29604]: Cluster is now healthy 2024-09-11T07:29:22.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:21 smithi143 ceph-mon[38451]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T07:29:22.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:21 smithi143 ceph-mon[38451]: Cluster is now healthy 2024-09-11T07:29:23.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:22 smithi164 ceph-mon[33167]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-11T07:29:23.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:22 smithi143 ceph-mon[29604]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-11T07:29:23.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:22 smithi143 ceph-mon[38451]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-11T07:29:24.909 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:24 smithi164 ceph-mon[33167]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T07:29:25.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:24 smithi143 ceph-mon[29604]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T07:29:25.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:24 smithi143 ceph-mon[38451]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T07:29:27.171 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:26 smithi164 ceph-mon[33167]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T07:29:27.172 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T07:29:27.172 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:26 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:27.172 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:26 smithi164 ceph-mon[33167]: Deploying daemon osd.7 on smithi164 2024-09-11T07:29:27.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:26 smithi143 ceph-mon[29604]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T07:29:27.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T07:29:27.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:26 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:27.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:26 smithi143 ceph-mon[29604]: Deploying daemon osd.7 on smithi164 2024-09-11T07:29:27.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:26 smithi143 ceph-mon[38451]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-11T07:29:27.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T07:29:27.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:26 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:27.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:26 smithi143 ceph-mon[38451]: Deploying daemon osd.7 on smithi164 2024-09-11T07:29:29.130 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:28 smithi164 ceph-mon[33167]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:29:29.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:28 smithi143 ceph-mon[29604]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:29:29.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:28 smithi143 ceph-mon[38451]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:29:30.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:30 smithi164 ceph-mon[33167]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:29:30.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:30 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:30.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:30 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:30.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:30 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:31.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:30 smithi143 ceph-mon[29604]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:29:31.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:30 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:31.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:30 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:31.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:30 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:31.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:30 smithi143 ceph-mon[38451]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:29:31.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:30 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:31.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:30 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:31.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:30 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:33.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:32 smithi164 ceph-mon[33167]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:33.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:32 smithi143 ceph-mon[29604]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:33.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:32 smithi143 ceph-mon[38451]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:34.292 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 7 on host 'smithi164' 2024-09-11T07:29:34.942 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:34 smithi164 ceph-mon[33167]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:34.942 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:34 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:34.943 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:34 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:34.943 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:34 smithi164 ceph-mon[33167]: from='osd.7 v2:172.21.15.164:6812/1031390908' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:29:34.943 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:34 smithi164 ceph-mon[33167]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:29:34.943 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:34 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:34.943 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:34 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:34.966 DEBUG:teuthology.orchestra.run.smithi164:osd.7> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T07:29:34.969 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-11T07:29:34.969 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T07:29:35.223 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:35.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:34 smithi143 ceph-mon[29604]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:35.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:34 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:35.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:34 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:35.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:34 smithi143 ceph-mon[29604]: from='osd.7 v2:172.21.15.164:6812/1031390908' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:29:35.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:34 smithi143 ceph-mon[29604]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:29:35.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:34 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:35.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:34 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:35.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:34 smithi143 ceph-mon[38451]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:35.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:34 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:35.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:34 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:35.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:34 smithi143 ceph-mon[38451]: from='osd.7 v2:172.21.15.164:6812/1031390908' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:29:35.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:34 smithi143 ceph-mon[38451]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:29:35.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:34 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:35.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:34 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:36.161 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:35 smithi164 ceph-mon[33167]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T07:29:36.161 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:35 smithi164 ceph-mon[33167]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T07:29:36.161 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:35 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:29:36.161 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:35 smithi164 ceph-mon[33167]: from='osd.7 v2:172.21.15.164:6812/1031390908' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:29:36.161 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:35 smithi164 ceph-mon[33167]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:29:36.161 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:29:35 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:29:35.908+0000 7ff53b2cd640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T07:29:36.162 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:29:35 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:29:35.915+0000 7ff535903640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:29:36.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:35 smithi143 ceph-mon[29604]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T07:29:36.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:35 smithi143 ceph-mon[29604]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T07:29:36.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:35 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:29:36.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:35 smithi143 ceph-mon[29604]: from='osd.7 v2:172.21.15.164:6812/1031390908' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:29:36.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:35 smithi143 ceph-mon[29604]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:29:36.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:35 smithi143 ceph-mon[38451]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T07:29:36.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:35 smithi143 ceph-mon[38451]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T07:29:36.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:35 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:29:36.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:35 smithi143 ceph-mon[38451]: from='osd.7 v2:172.21.15.164:6812/1031390908' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:29:36.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:35 smithi143 ceph-mon[38451]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-11T07:29:37.041 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:29:37.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:36 smithi164 ceph-mon[33167]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:37.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:36 smithi164 ceph-mon[33167]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-11T07:29:37.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:36 smithi164 ceph-mon[33167]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T07:29:37.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:36 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:29:37.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:36 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:29:37.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:36 smithi143 ceph-mon[29604]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:37.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:36 smithi143 ceph-mon[29604]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-11T07:29:37.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:36 smithi143 ceph-mon[29604]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T07:29:37.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:36 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:29:37.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:36 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:29:37.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:36 smithi143 ceph-mon[38451]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:29:37.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:36 smithi143 ceph-mon[38451]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-11T07:29:37.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:36 smithi143 ceph-mon[38451]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T07:29:37.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:36 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:29:37.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:36 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:29:37.713 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":49,"num_osds":8,"num_up_osds":8,"osd_up_since":1726039776,"num_in_osds":8,"osd_in_since":1726039760,"num_remapped_pgs":1} 2024-09-11T07:29:37.715 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T07:29:37.953 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: purged_snaps scrub starts 2024-09-11T07:29:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: purged_snaps scrub ok 2024-09-11T07:29:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: osd.7 v2:172.21.15.164:6812/1031390908 boot 2024-09-11T07:29:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T07:29:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:29:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1468635095' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T07:29:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:38.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:38.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: purged_snaps scrub starts 2024-09-11T07:29:38.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: purged_snaps scrub ok 2024-09-11T07:29:38.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: osd.7 v2:172.21.15.164:6812/1031390908 boot 2024-09-11T07:29:38.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T07:29:38.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:29:38.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1468635095' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T07:29:38.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:38.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:38.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:38.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:38.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:38.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:38.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:38.052 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:38.052 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:38.052 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:38.052 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:38.052 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:38.052 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:38.052 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:37 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:38.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: purged_snaps scrub starts 2024-09-11T07:29:38.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: purged_snaps scrub ok 2024-09-11T07:29:38.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: osd.7 v2:172.21.15.164:6812/1031390908 boot 2024-09-11T07:29:38.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T07:29:38.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:29:38.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1468635095' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T07:29:38.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:38.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:38.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:38.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:38.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:38.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:29:38.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:38.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:38.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:38.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:38.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:38.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:38.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:38.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:37 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:39.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:38 smithi143 ceph-mon[38451]: Detected new or changed devices on smithi164 2024-09-11T07:29:39.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:38 smithi143 ceph-mon[38451]: Adjusting osd_memory_target on smithi164 to 4272M 2024-09-11T07:29:39.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:38 smithi143 ceph-mon[38451]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:39.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:38 smithi143 ceph-mon[38451]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T07:29:39.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:38 smithi143 ceph-mon[29604]: Detected new or changed devices on smithi164 2024-09-11T07:29:39.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:38 smithi143 ceph-mon[29604]: Adjusting osd_memory_target on smithi164 to 4272M 2024-09-11T07:29:39.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:38 smithi143 ceph-mon[29604]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:39.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:38 smithi143 ceph-mon[29604]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T07:29:39.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:38 smithi164 ceph-mon[33167]: Detected new or changed devices on smithi164 2024-09-11T07:29:39.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:38 smithi164 ceph-mon[33167]: Adjusting osd_memory_target on smithi164 to 4272M 2024-09-11T07:29:39.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:38 smithi164 ceph-mon[33167]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:39.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:38 smithi164 ceph-mon[33167]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T07:29:39.715 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:29:39.715 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":51,"fsid":"d3dfd926-700e-11ef-bcea-c7b262605968","created":"2024-09-11T07:24:43.922426+0000","modified":"2024-09-11T07:29:38.908841+0000","last_up_change":"2024-09-11T07:29:36.902426+0000","last_in_change":"2024-09-11T07:29:20.147581+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:27:45.934203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"aafa4449-5343-4ce4-bb8a-1935f5e4fcbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6801","nonce":168444810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":168444810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":168444810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":168444810}]},"public_addr":"172.21.15.143:6801/168444810","cluster_addr":"172.21.15.143:6802/168444810","heartbeat_back_addr":"172.21.15.143:6804/168444810","heartbeat_front_addr":"172.21.15.143:6803/168444810","state":["exists","up"]},{"osd":1,"uuid":"633a8442-c551-4678-bd79-d938e83c11cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6805","nonce":507789371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":507789371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":507789371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":507789371}]},"public_addr":"172.21.15.143:6805/507789371","cluster_addr":"172.21.15.143:6806/507789371","heartbeat_back_addr":"172.21.15.143:6808/507789371","heartbeat_front_addr":"172.21.15.143:6807/507789371","state":["exists","up"]},{"osd":2,"uuid":"0cad9b00-6cc8-4cde-abe0-64f80bccb80b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6809","nonce":2660767941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2660767941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":2660767941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2660767941}]},"public_addr":"172.21.15.143:6809/2660767941","cluster_addr":"172.21.15.143:6810/2660767941","heartbeat_back_addr":"172.21.15.143:6812/2660767941","heartbeat_front_addr":"172.21.15.143:6811/2660767941","state":["exists","up"]},{"osd":3,"uuid":"b4a7c5e6-6afd-4bfc-8db9-ee6c4349061f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":2376878046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2376878046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2376878046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6815","nonce":2376878046}]},"public_addr":"172.21.15.143:6813/2376878046","cluster_addr":"172.21.15.143:6814/2376878046","heartbeat_back_addr":"172.21.15.143:6816/2376878046","heartbeat_front_addr":"172.21.15.143:6815/2376878046","state":["exists","up"]},{"osd":4,"uuid":"61aa7852-9e2a-4498-9f68-b248012431a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2286891940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6801","nonce":2286891940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6803","nonce":2286891940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2286891940}]},"public_addr":"172.21.15.164:6800/2286891940","cluster_addr":"172.21.15.164:6801/2286891940","heartbeat_back_addr":"172.21.15.164:6803/2286891940","heartbeat_front_addr":"172.21.15.164:6802/2286891940","state":["exists","up"]},{"osd":5,"uuid":"bd3428b7-b0c6-474e-b462-d964c1b91327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4094877797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6805","nonce":4094877797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6807","nonce":4094877797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4094877797}]},"public_addr":"172.21.15.164:6804/4094877797","cluster_addr":"172.21.15.164:6805/4094877797","heartbeat_back_addr":"172.21.15.164:6807/4094877797","heartbeat_front_addr":"172.21.15.164:6806/4094877797","state":["exists","up"]},{"osd":6,"uuid":"abbb121a-3d68-4661-8087-f8630232871e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":174920052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":174920052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6811","nonce":174920052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":174920052}]},"public_addr":"172.21.15.164:6808/174920052","cluster_addr":"172.21.15.164:6809/174920052","heartbeat_back_addr":"172.21.15.164:6811/174920052","heartbeat_front_addr":"172.21.15.164:6810/174920052","state":["exists","up"]},{"osd":7,"uuid":"f3b1d8ce-1a3a-4da5-bcf2-46d1aeb38b95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1031390908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":1031390908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6815","nonce":1031390908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1031390908}]},"public_addr":"172.21.15.164:6812/1031390908","cluster_addr":"172.21.15.164:6813/1031390908","heartbeat_back_addr":"172.21.15.164:6815/1031390908","heartbeat_front_addr":"172.21.15.164:6814/1031390908","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:56.642492+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:27:19.610748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:27:42.158678+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:28:05.261976+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:28:26.646599+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:28:49.900951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:29:12.845103+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:29:34.961584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/4260814966":"2024-09-12T07:25:37.628302+0000","172.21.15.143:0/3068436287":"2024-09-12T07:25:37.628302+0000","172.21.15.143:0/2077730301":"2024-09-12T07:25:15.340448+0000","172.21.15.143:0/105787121":"2024-09-12T07:25:15.340448+0000","172.21.15.143:0/1457644251":"2024-09-12T07:25:15.340448+0000","172.21.15.143:0/2914000301":"2024-09-12T07:25:37.628302+0000","172.21.15.143:6800/2276713308":"2024-09-12T07:25:37.628302+0000","172.21.15.143:6800/1120431981":"2024-09-12T07:25:15.340448+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T07:29:40.023 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:39 smithi143 ceph-mon[29604]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T07:29:40.024 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:39 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/228755450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:29:40.024 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:39 smithi143 ceph-mon[38451]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T07:29:40.024 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:39 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/228755450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:29:40.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:39 smithi164 ceph-mon[33167]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T07:29:40.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:39 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/228755450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:29:40.381 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T07:27:45.934203+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-11T07:29:40.382 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T07:29:40.628 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:41.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:40 smithi143 ceph-mon[29604]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:41.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:40 smithi143 ceph-mon[38451]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:41.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:40 smithi164 ceph-mon[33167]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:42.514 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-09-11T07:29:43.163 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:42 smithi143 ceph-mon[38451]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:43.163 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:42 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1296290318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T07:29:43.163 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:42 smithi143 ceph-mon[29604]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:43.163 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:42 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1296290318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T07:29:43.187 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi143 2024-09-11T07:29:43.188 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi143=foo.a' 2024-09-11T07:29:43.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:42 smithi164 ceph-mon[33167]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:43.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:42 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1296290318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T07:29:43.452 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:29:45.176 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled rgw.foo.a update... 2024-09-11T07:29:45.177 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:44 smithi164 ceph-mon[33167]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:45.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:44 smithi143 ceph-mon[29604]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:45.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:44 smithi143 ceph-mon[38451]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:45.791 DEBUG:teuthology.orchestra.run.smithi143:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-11T07:29:45.794 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi164 2024-09-11T07:29:45.794 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-11T07:29:46.074 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:29:46.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:46 smithi164 ceph-mon[33167]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi143=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:29:46.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:46 smithi164 ceph-mon[33167]: Saving service rgw.foo.a spec with placement smithi143=foo.a;count:1 2024-09-11T07:29:46.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:46 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:46.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:46 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:46.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:46 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:46.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:46 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:46.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:46 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:46.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:46 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T07:29:46.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:46 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T07:29:46.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:46 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:46.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:46 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:46.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:46 smithi164 ceph-mon[33167]: Deploying daemon rgw.foo.a on smithi143 2024-09-11T07:29:46.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[38451]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi143=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:29:46.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[38451]: Saving service rgw.foo.a spec with placement smithi143=foo.a;count:1 2024-09-11T07:29:46.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:46.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:46.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:46.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:46.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:46.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T07:29:46.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T07:29:46.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:46.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:46.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[38451]: Deploying daemon rgw.foo.a on smithi143 2024-09-11T07:29:46.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[29604]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi143=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:29:46.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[29604]: Saving service rgw.foo.a spec with placement smithi143=foo.a;count:1 2024-09-11T07:29:46.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:46.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:46.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:46.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:46.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:46.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T07:29:46.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T07:29:46.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:46.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:46.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:46 smithi143 ceph-mon[29604]: Deploying daemon rgw.foo.a on smithi143 2024-09-11T07:29:47.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:47 smithi143 ceph-mon[29604]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:47.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:47 smithi143 ceph-mon[38451]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:47.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:47 smithi164 ceph-mon[33167]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:47.798 INFO:journalctl@ceph.rgw.foo.a.smithi143.stdout:Sep 11 07:29:47 smithi143 systemd[1]: Started Ceph rgw.foo.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:29:48.210 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'datapool' created 2024-09-11T07:29:48.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:48 smithi164 ceph-mon[33167]: from='client.? 172.21.15.164:0/1565731369' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:29:48.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:48 smithi164 ceph-mon[33167]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:29:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:48 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:48 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:48 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:48 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:48 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:48.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:48 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:48.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:48 smithi143 ceph-mon[29604]: from='client.? 172.21.15.164:0/1565731369' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:29:48.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:48 smithi143 ceph-mon[29604]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:29:48.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:48 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:48.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:48 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:48.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:48 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:48.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:48 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:48.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:48 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:48.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:48 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:48.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:48 smithi143 ceph-mon[38451]: from='client.? 172.21.15.164:0/1565731369' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:29:48.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:48 smithi143 ceph-mon[38451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:29:48.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:48 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:48.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:48 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:48.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:48 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:48.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:48 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:48.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:48 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:48.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:48 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:48.730 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-11T07:29:48.973 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:29:49.350 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:49 smithi164 ceph-mon[33167]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:49.350 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:49 smithi164 ceph-mon[33167]: Saving service rgw.foo.a spec with placement smithi143=foo.a;count:1 2024-09-11T07:29:49.350 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:49 smithi164 ceph-mon[33167]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T07:29:49.351 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:49 smithi164 ceph-mon[33167]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T07:29:49.351 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:49 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2306460240' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:29:49.351 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:49 smithi164 ceph-mon[33167]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:29:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:49 smithi143 ceph-mon[29604]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:49 smithi143 ceph-mon[29604]: Saving service rgw.foo.a spec with placement smithi143=foo.a;count:1 2024-09-11T07:29:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:49 smithi143 ceph-mon[29604]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T07:29:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:49 smithi143 ceph-mon[29604]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T07:29:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:49 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2306460240' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:29:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:49 smithi143 ceph-mon[29604]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:29:49.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:49 smithi143 ceph-mon[38451]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:49.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:49 smithi143 ceph-mon[38451]: Saving service rgw.foo.a spec with placement smithi143=foo.a;count:1 2024-09-11T07:29:49.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:49 smithi143 ceph-mon[38451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T07:29:49.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:49 smithi143 ceph-mon[38451]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T07:29:49.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:49 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2306460240' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:29:49.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:49 smithi143 ceph-mon[38451]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:29:50.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:50 smithi143 ceph-mon[29604]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T07:29:50.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:50 smithi143 ceph-mon[29604]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T07:29:50.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:50 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:50.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:50 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:50.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:50 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:50.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:50 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:50.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:50 smithi143 ceph-mon[38451]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T07:29:50.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:50 smithi143 ceph-mon[38451]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T07:29:50.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:50 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:50.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:50 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:50.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:50 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:50.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:50 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:50.352 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:50 smithi164 ceph-mon[33167]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T07:29:50.352 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:50 smithi164 ceph-mon[33167]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T07:29:50.352 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:50 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:50.352 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:50 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:50.352 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:50 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:50.352 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:50 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[38451]: Checking dashboard <-> RGW credentials 2024-09-11T07:29:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[38451]: pgmap v169: 36 pgs: 5 creating+peering, 27 unknown, 4 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[38451]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T07:29:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3945602096' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:29:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2306460240' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:29:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[38451]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:29:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[38451]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:29:51.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[38451]: from='client.? 172.21.15.164:0/2752541880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:29:51.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[38451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:29:51.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[38451]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:29:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[29604]: Checking dashboard <-> RGW credentials 2024-09-11T07:29:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[29604]: pgmap v169: 36 pgs: 5 creating+peering, 27 unknown, 4 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[29604]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T07:29:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3945602096' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:29:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2306460240' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:29:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[29604]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:29:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[29604]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:29:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[29604]: from='client.? 172.21.15.164:0/2752541880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:29:51.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[29604]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:29:51.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:51 smithi143 ceph-mon[29604]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:29:51.637 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:51 smithi164 ceph-mon[33167]: Checking dashboard <-> RGW credentials 2024-09-11T07:29:51.637 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:51 smithi164 ceph-mon[33167]: pgmap v169: 36 pgs: 5 creating+peering, 27 unknown, 4 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:51.637 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:51 smithi164 ceph-mon[33167]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T07:29:51.637 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:51 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3945602096' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:29:51.637 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:51 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2306460240' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:29:51.638 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:51 smithi164 ceph-mon[33167]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:29:51.638 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:51 smithi164 ceph-mon[33167]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:29:51.638 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:51 smithi164 ceph-mon[33167]: from='client.? 172.21.15.164:0/2752541880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:29:51.638 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:51 smithi164 ceph-mon[33167]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:29:51.638 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:51 smithi164 ceph-mon[33167]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:29:52.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:52 smithi143 ceph-mon[29604]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:29:52.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:52 smithi143 ceph-mon[29604]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:29:52.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:52 smithi143 ceph-mon[29604]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T07:29:52.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:52 smithi143 ceph-mon[29604]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T07:29:52.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:52 smithi143 ceph-mon[38451]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:29:52.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:52 smithi143 ceph-mon[38451]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:29:52.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:52 smithi143 ceph-mon[38451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T07:29:52.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:52 smithi143 ceph-mon[38451]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T07:29:52.596 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:52 smithi164 ceph-mon[33167]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:29:52.597 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:52 smithi164 ceph-mon[33167]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:29:52.597 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:52 smithi164 ceph-mon[33167]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T07:29:52.597 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:52 smithi164 ceph-mon[33167]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T07:29:53.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:53 smithi164 ceph-mon[33167]: pgmap v172: 68 pgs: 5 creating+peering, 50 unknown, 13 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:53.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:53 smithi164 ceph-mon[33167]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:29:53.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:53 smithi164 ceph-mon[33167]: Cluster is now healthy 2024-09-11T07:29:53.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:53 smithi164 ceph-mon[33167]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T07:29:53.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:53 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3945602096' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:29:53.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:53 smithi164 ceph-mon[33167]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:29:53.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:53 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2306460240' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:29:53.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:53 smithi164 ceph-mon[33167]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:29:53.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:53 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:53.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[29604]: pgmap v172: 68 pgs: 5 creating+peering, 50 unknown, 13 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:53.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[29604]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:29:53.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[29604]: Cluster is now healthy 2024-09-11T07:29:53.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[29604]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T07:29:53.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3945602096' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:29:53.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[29604]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:29:53.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2306460240' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:29:53.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[29604]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:29:53.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:53.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[38451]: pgmap v172: 68 pgs: 5 creating+peering, 50 unknown, 13 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:29:53.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[38451]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:29:53.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[38451]: Cluster is now healthy 2024-09-11T07:29:53.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[38451]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T07:29:53.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3945602096' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:29:53.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[38451]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:29:53.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2306460240' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:29:53.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[38451]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:29:53.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:53 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:53.792 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.164 --placement '1;smithi164=iscsi.a' 2024-09-11T07:29:54.038 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:29:54.411 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:54 smithi164 ceph-mon[33167]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:29:54.412 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:54 smithi164 ceph-mon[33167]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:29:54.412 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:54 smithi164 ceph-mon[33167]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T07:29:54.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:54 smithi143 ceph-mon[29604]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:29:54.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:54 smithi143 ceph-mon[29604]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:29:54.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:54 smithi143 ceph-mon[29604]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T07:29:54.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:54 smithi143 ceph-mon[38451]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:29:54.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:54 smithi143 ceph-mon[38451]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:29:54.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:54 smithi143 ceph-mon[38451]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T07:29:55.310 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:55 smithi164 ceph-mon[33167]: pgmap v175: 100 pgs: 1 creating+peering, 46 unknown, 53 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T07:29:55.310 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:55 smithi164 ceph-mon[33167]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T07:29:55.310 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:55 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3777511412' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:29:55.310 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:55 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1904941676' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:29:55.310 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:55 smithi164 ceph-mon[33167]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:29:55.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:55 smithi143 ceph-mon[29604]: pgmap v175: 100 pgs: 1 creating+peering, 46 unknown, 53 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T07:29:55.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:55 smithi143 ceph-mon[29604]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T07:29:55.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:55 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3777511412' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:29:55.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:55 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1904941676' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:29:55.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:55 smithi143 ceph-mon[29604]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:29:55.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:55 smithi143 ceph-mon[38451]: pgmap v175: 100 pgs: 1 creating+peering, 46 unknown, 53 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T07:29:55.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:55 smithi143 ceph-mon[38451]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T07:29:55.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:55 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3777511412' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:29:55.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:55 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1904941676' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:29:55.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:55 smithi143 ceph-mon[38451]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:29:55.682 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled iscsi.datapool update... 2024-09-11T07:29:56.510 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-11T07:29:56.511 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-11T07:29:56.511 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T07:29:56.536 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1904941676' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:29:56.536 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[29604]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:29:56.536 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[29604]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T07:29:56.536 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1904941676' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:29:56.536 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3777511412' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:29:56.536 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[29604]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:29:56.536 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:56.536 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1904941676' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:29:56.536 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[29604]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:29:56.537 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[29604]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T07:29:56.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1904941676' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:29:56.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[38451]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:29:56.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[38451]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T07:29:56.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1904941676' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:29:56.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3777511412' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:29:56.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[38451]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:29:56.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:56.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1904941676' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:29:56.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[38451]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:29:56.537 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:56 smithi143 ceph-mon[38451]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T07:29:56.540 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-11T07:29:56.541 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T07:29:56.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:56 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1904941676' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:29:56.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:56 smithi164 ceph-mon[33167]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:29:56.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:56 smithi164 ceph-mon[33167]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T07:29:56.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:56 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1904941676' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:29:56.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:56 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3777511412' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:29:56.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:56 smithi164 ceph-mon[33167]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:29:56.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:56 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:56.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:56 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1904941676' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:29:56.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:56 smithi164 ceph-mon[33167]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:29:56.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:56 smithi164 ceph-mon[33167]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T07:29:56.570 DEBUG:teuthology.orchestra.run.smithi164:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-11T07:29:56.612 INFO:tasks.cephadm:Adding prometheus.a on smithi164 2024-09-11T07:29:56.612 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi164=a' 2024-09-11T07:29:57.451 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:29:57.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:57 smithi164 ceph-mon[33167]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.164", "placement": "1;smithi164=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:29:57.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:57 smithi164 ceph-mon[33167]: Saving service iscsi.datapool spec with placement smithi164=iscsi.a;count:1 2024-09-11T07:29:57.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:57 smithi164 ceph-mon[33167]: pgmap v178: 132 pgs: 7 creating+peering, 38 unknown, 87 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-11T07:29:57.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:57 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:57.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:57 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:57.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:57 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:57.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:57 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T07:29:57.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:57 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T07:29:57.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:57 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:57.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:57 smithi164 ceph-mon[33167]: Deploying daemon iscsi.iscsi.a on smithi164 2024-09-11T07:29:57.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[29604]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.164", "placement": "1;smithi164=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:29:57.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[29604]: Saving service iscsi.datapool spec with placement smithi164=iscsi.a;count:1 2024-09-11T07:29:57.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[29604]: pgmap v178: 132 pgs: 7 creating+peering, 38 unknown, 87 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-11T07:29:57.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:57.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:57.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:57.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T07:29:57.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T07:29:57.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:57.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[29604]: Deploying daemon iscsi.iscsi.a on smithi164 2024-09-11T07:29:57.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[38451]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.164", "placement": "1;smithi164=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:29:57.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[38451]: Saving service iscsi.datapool spec with placement smithi164=iscsi.a;count:1 2024-09-11T07:29:57.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[38451]: pgmap v178: 132 pgs: 7 creating+peering, 38 unknown, 87 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-11T07:29:57.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:57.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:57.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:29:57.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T07:29:57.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T07:29:57.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:57.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:57 smithi143 ceph-mon[38451]: Deploying daemon iscsi.iscsi.a on smithi164 2024-09-11T07:29:58.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:29:58 smithi164 ceph-mon[33167]: pgmap v180: 132 pgs: 7 creating+peering, 12 unknown, 113 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 30 KiB/s rd, 4.0 KiB/s wr, 70 op/s 2024-09-11T07:29:58.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:29:58 smithi143 ceph-mon[29604]: pgmap v180: 132 pgs: 7 creating+peering, 12 unknown, 113 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 30 KiB/s rd, 4.0 KiB/s wr, 70 op/s 2024-09-11T07:29:58.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:29:58 smithi143 ceph-mon[38451]: pgmap v180: 132 pgs: 7 creating+peering, 12 unknown, 113 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 30 KiB/s rd, 4.0 KiB/s wr, 70 op/s 2024-09-11T07:29:59.609 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled prometheus update... 2024-09-11T07:30:00.201 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:00 smithi164 ceph-mon[33167]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T07:30:00.201 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:00 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:00.201 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:00 smithi164 ceph-mon[33167]: overall HEALTH_OK 2024-09-11T07:30:00.210 DEBUG:teuthology.orchestra.run.smithi164:prometheus.a> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-11T07:30:00.213 INFO:tasks.cephadm:Adding node-exporter.a on smithi143 2024-09-11T07:30:00.213 INFO:tasks.cephadm:Adding node-exporter.b on smithi164 2024-09-11T07:30:00.213 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi143=a;smithi164=b' 2024-09-11T07:30:00.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:00 smithi143 ceph-mon[29604]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T07:30:00.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:00 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:00.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:00 smithi143 ceph-mon[29604]: overall HEALTH_OK 2024-09-11T07:30:00.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:00 smithi143 ceph-mon[38451]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T07:30:00.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:00 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:00.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:00 smithi143 ceph-mon[38451]: overall HEALTH_OK 2024-09-11T07:30:00.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:00 smithi164 podman[62900]: 2024-09-11 07:30:00.587910592 +0000 UTC m=+0.221917751 container create 10287ccb5bb223eab77b148be2f3d11d450f7cc6c1dbb343176cc9350a601218 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T07:30:00.990 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:30:01.113 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:00 smithi164 podman[62900]: 2024-09-11 07:30:00.831602076 +0000 UTC m=+0.465609195 container init 10287ccb5bb223eab77b148be2f3d11d450f7cc6c1dbb343176cc9350a601218 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:30:01.113 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:00 smithi164 podman[62900]: 2024-09-11 07:30:00.836607455 +0000 UTC m=+0.470614573 container start 10287ccb5bb223eab77b148be2f3d11d450f7cc6c1dbb343176cc9350a601218 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:30:01.114 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:00 smithi164 bash[62900]: 10287ccb5bb223eab77b148be2f3d11d450f7cc6c1dbb343176cc9350a601218 2024-09-11T07:30:01.114 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:00 smithi164 systemd[1]: Started Ceph iscsi.iscsi.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:30:01.369 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug Started the configuration object watcher 2024-09-11T07:30:01.369 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug Checking for config object changes every 1s 2024-09-11T07:30:01.369 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug Processing osd blocklist entries for this node 2024-09-11T07:30:01.370 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-mon[33167]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi164=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:30:01.370 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-mon[33167]: Saving service prometheus spec with placement smithi164=a;count:1 2024-09-11T07:30:01.370 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-mon[33167]: pgmap v182: 132 pgs: 2 creating+peering, 130 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 74 KiB/s rd, 6.1 KiB/s wr, 178 op/s 2024-09-11T07:30:01.370 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:01.370 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:01.370 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:01.370 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:01.370 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:30:01.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:01 smithi143 ceph-mon[29604]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi164=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:30:01.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:01 smithi143 ceph-mon[29604]: Saving service prometheus spec with placement smithi164=a;count:1 2024-09-11T07:30:01.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:01 smithi143 ceph-mon[29604]: pgmap v182: 132 pgs: 2 creating+peering, 130 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 74 KiB/s rd, 6.1 KiB/s wr, 178 op/s 2024-09-11T07:30:01.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:01 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:01.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:01 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:01.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:01 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:01.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:01 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:01.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:01 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:30:01.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:01 smithi143 ceph-mon[38451]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi164=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:30:01.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:01 smithi143 ceph-mon[38451]: Saving service prometheus spec with placement smithi164=a;count:1 2024-09-11T07:30:01.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:01 smithi143 ceph-mon[38451]: pgmap v182: 132 pgs: 2 creating+peering, 130 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 74 KiB/s rd, 6.1 KiB/s wr, 178 op/s 2024-09-11T07:30:01.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:01 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:01.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:01 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:01.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:01 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:01.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:01 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:01.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:01 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:30:01.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug Reading the configuration object to update local LIO configuration 2024-09-11T07:30:01.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug Configuration does not have an entry for this host(smithi164.front.sepia.ceph.com) - nothing to define to LIO 2024-09-11T07:30:01.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-11T07:30:01.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: * Environment: production 2024-09-11T07:30:01.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T07:30:01.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: Use a production WSGI server instead. 2024-09-11T07:30:01.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: * Debug mode: off 2024-09-11T07:30:01.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug * Running on all addresses. 2024-09-11T07:30:01.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T07:30:01.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: * Running on all addresses. 2024-09-11T07:30:01.718 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T07:30:01.718 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug * Running on http://[fe80::ec4:7aff:fe88:73f3]:5000/ (Press CTRL+C to quit) 2024-09-11T07:30:01.718 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: * Running on http://[fe80::ec4:7aff:fe88:73f3]:5000/ (Press CTRL+C to quit) 2024-09-11T07:30:02.278 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:02 smithi164 ceph-mon[33167]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T07:30:02.278 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:02 smithi164 ceph-mon[33167]: from='client.? 172.21.15.164:0/1111599263' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T07:30:02.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:02 smithi143 ceph-mon[29604]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T07:30:02.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:02 smithi143 ceph-mon[29604]: from='client.? 172.21.15.164:0/1111599263' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T07:30:02.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:02 smithi143 ceph-mon[38451]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T07:30:02.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:02 smithi143 ceph-mon[38451]: from='client.? 172.21.15.164:0/1111599263' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T07:30:02.657 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled node-exporter update... 2024-09-11T07:30:03.349 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:03 smithi164 ceph-mon[33167]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s rd, 5.4 KiB/s wr, 174 op/s 2024-09-11T07:30:03.349 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:03 smithi164 ceph-mon[33167]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T07:30:03.349 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:03 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:03.349 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:03 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:03.372 DEBUG:teuthology.orchestra.run.smithi143:node-exporter.a> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-11T07:30:03.374 DEBUG:teuthology.orchestra.run.smithi164:node-exporter.b> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-11T07:30:03.377 INFO:tasks.cephadm:Adding alertmanager.a on smithi143 2024-09-11T07:30:03.377 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi143=a' 2024-09-11T07:30:03.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:03 smithi143 ceph-mon[29604]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s rd, 5.4 KiB/s wr, 174 op/s 2024-09-11T07:30:03.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:03 smithi143 ceph-mon[29604]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T07:30:03.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:03 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:03.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:03 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:03 smithi143 ceph-mon[38451]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s rd, 5.4 KiB/s wr, 174 op/s 2024-09-11T07:30:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:03 smithi143 ceph-mon[38451]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T07:30:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:03 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:03 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:03.846 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:30:04.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:04 smithi164 ceph-mon[33167]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi143=a;smithi164=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:30:04.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:04 smithi164 ceph-mon[33167]: Saving service node-exporter spec with placement smithi143=a;smithi164=b;count:2 2024-09-11T07:30:04.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:04 smithi143 ceph-mon[29604]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi143=a;smithi164=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:30:04.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:04 smithi143 ceph-mon[29604]: Saving service node-exporter spec with placement smithi143=a;smithi164=b;count:2 2024-09-11T07:30:04.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:04 smithi143 ceph-mon[38451]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi143=a;smithi164=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:30:04.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:04 smithi143 ceph-mon[38451]: Saving service node-exporter spec with placement smithi143=a;smithi164=b;count:2 2024-09-11T07:30:05.359 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:05 smithi164 ceph-mon[33167]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-09-11T07:30:05.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:05 smithi143 ceph-mon[29604]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-09-11T07:30:05.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:05 smithi143 ceph-mon[38451]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-09-11T07:30:06.003 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled alertmanager update... 2024-09-11T07:30:06.582 DEBUG:teuthology.orchestra.run.smithi143:alertmanager.a> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-11T07:30:06.586 INFO:tasks.cephadm:Adding grafana.a on smithi164 2024-09-11T07:30:06.586 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi164=a' 2024-09-11T07:30:06.830 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:30:06.950 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:06 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:06.950 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:06 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:06.950 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:06 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:30:06.951 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:06 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:30:06.951 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:06 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:06.951 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:06 smithi164 ceph-mon[33167]: Deploying daemon prometheus.a on smithi164 2024-09-11T07:30:06.951 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:06 smithi164 ceph-mon[33167]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 3.8 KiB/s wr, 119 op/s 2024-09-11T07:30:06.951 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:06 smithi164 ceph-mon[33167]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi143=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:30:06.951 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:06 smithi164 ceph-mon[33167]: Saving service alertmanager spec with placement smithi143=a;count:1 2024-09-11T07:30:06.951 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:06 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:07.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:07.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:07.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:30:07.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:30:07.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:07.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[29604]: Deploying daemon prometheus.a on smithi164 2024-09-11T07:30:07.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[29604]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 3.8 KiB/s wr, 119 op/s 2024-09-11T07:30:07.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[29604]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi143=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:30:07.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[29604]: Saving service alertmanager spec with placement smithi143=a;count:1 2024-09-11T07:30:07.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:07.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:07.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:07.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:30:07.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:30:07.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:07.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[38451]: Deploying daemon prometheus.a on smithi164 2024-09-11T07:30:07.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[38451]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 3.8 KiB/s wr, 119 op/s 2024-09-11T07:30:07.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[38451]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi143=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:30:07.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[38451]: Saving service alertmanager spec with placement smithi143=a;count:1 2024-09-11T07:30:07.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:06 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:08.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:08 smithi164 ceph-mon[33167]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 34 KiB/s rd, 2.3 KiB/s wr, 84 op/s 2024-09-11T07:30:08.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:08 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:09.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:08 smithi143 ceph-mon[29604]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 34 KiB/s rd, 2.3 KiB/s wr, 84 op/s 2024-09-11T07:30:09.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:08 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:09.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:08 smithi143 ceph-mon[38451]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 34 KiB/s rd, 2.3 KiB/s wr, 84 op/s 2024-09-11T07:30:09.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:08 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:09.184 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled grafana update... 2024-09-11T07:30:10.021 DEBUG:teuthology.orchestra.run.smithi164:grafana.a> sudo journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@grafana.a.service 2024-09-11T07:30:10.023 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T07:30:10.024 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T07:30:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:10.398 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:10 smithi143 ceph-mon[29604]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi164=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:30:10.398 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:10 smithi143 ceph-mon[29604]: Saving service grafana spec with placement smithi164=a;count:1 2024-09-11T07:30:10.398 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:10 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:10.398 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:10 smithi143 ceph-mon[38451]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi164=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:30:10.398 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:10 smithi143 ceph-mon[38451]: Saving service grafana spec with placement smithi164=a;count:1 2024-09-11T07:30:10.398 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:10 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:10.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:10 smithi164 ceph-mon[33167]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi164=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:30:10.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:10 smithi164 ceph-mon[33167]: Saving service grafana spec with placement smithi164=a;count:1 2024-09-11T07:30:10.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:10 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:11.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:11 smithi164 ceph-mon[33167]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 13 KiB/s rd, 878 B/s wr, 32 op/s 2024-09-11T07:30:11.467 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:30:11.487 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:11 smithi143 ceph-mon[29604]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 13 KiB/s rd, 878 B/s wr, 32 op/s 2024-09-11T07:30:11.487 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:11 smithi143 ceph-mon[38451]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 13 KiB/s rd, 878 B/s wr, 32 op/s 2024-09-11T07:30:12.335 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2024-09-11T07:30:12.335 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAER+Fm9NqsExAAAqfPjLzyR5ocCV4apdIZ2Q== 2024-09-11T07:30:12.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:12 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:12.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:12 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:12.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:12 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:13.029 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-11T07:30:13.029 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T07:30:13.029 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T07:30:13.069 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T07:30:13.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:13 smithi143 ceph-mon[29604]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 341 B/s wr, 14 op/s 2024-09-11T07:30:13.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:13 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/71303682' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:30:13.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:13 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/71303682' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:30:13.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:13 smithi143 ceph-mon[38451]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 341 B/s wr, 14 op/s 2024-09-11T07:30:13.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:13 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/71303682' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:30:13.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:13 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/71303682' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:30:13.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:13 smithi164 ceph-mon[33167]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 341 B/s wr, 14 op/s 2024-09-11T07:30:13.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:13 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/71303682' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:30:13.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:13 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/71303682' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:30:13.974 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:30:15.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:15 smithi164 ceph-mon[33167]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T07:30:15.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:15 smithi143 ceph-mon[29604]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T07:30:15.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:15 smithi143 ceph-mon[38451]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T07:30:17.447 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:17 smithi164 ceph-mon[33167]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:30:17.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:17 smithi143 ceph-mon[29604]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:30:17.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:17 smithi143 ceph-mon[38451]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:30:18.799 INFO:teuthology.orchestra.run.smithi164.stdout:[client.1] 2024-09-11T07:30:18.799 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQAKR+Fm/JZOLxAAtS8n3znrSkimE5CRsFXGww== 2024-09-11T07:30:19.464 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:19 smithi164 ceph-mon[33167]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:30:19.464 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:19 smithi164 ceph-mon[33167]: from='client.? 172.21.15.164:0/756957910' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:30:19.465 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:19 smithi164 ceph-mon[33167]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:30:19.465 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:19 smithi164 ceph-mon[33167]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:30:19.465 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:19 smithi164 systemd[1]: Starting Ceph prometheus.a for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:30:19.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:19 smithi143 ceph-mon[38451]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:30:19.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:19 smithi143 ceph-mon[38451]: from='client.? 172.21.15.164:0/756957910' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:30:19.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:19 smithi143 ceph-mon[38451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:30:19.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:19 smithi143 ceph-mon[38451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:30:19.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:19 smithi143 ceph-mon[29604]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:30:19.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:19 smithi143 ceph-mon[29604]: from='client.? 172.21.15.164:0/756957910' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:30:19.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:19 smithi143 ceph-mon[29604]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:30:19.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:19 smithi143 ceph-mon[29604]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:30:19.869 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-11T07:30:19.869 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-11T07:30:19.869 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-11T07:30:19.908 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T07:30:19.908 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T07:30:19.908 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T07:30:20.160 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:20.360 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 podman[64832]: 2024-09-11 07:30:20.103145394 +0000 UTC m=+0.112467389 container create 55c09ced2ef69c97dd9ead9f397957cca3a365e6f8f301603a4c81d7ae8a9a1d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:30:20.361 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 podman[64832]: 2024-09-11 07:30:20.00828296 +0000 UTC m=+0.017604921 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-11T07:30:20.361 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 podman[64832]: 2024-09-11 07:30:20.305566617 +0000 UTC m=+0.314888576 container init 55c09ced2ef69c97dd9ead9f397957cca3a365e6f8f301603a4c81d7ae8a9a1d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:30:20.361 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 podman[64832]: 2024-09-11 07:30:20.310437245 +0000 UTC m=+0.319759196 container start 55c09ced2ef69c97dd9ead9f397957cca3a365e6f8f301603a4c81d7ae8a9a1d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:30:20.361 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.335Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T07:30:20.361 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.335Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T07:30:20.361 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.335Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi164 (none))" 2024-09-11T07:30:20.361 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.335Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T07:30:20.361 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.335Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T07:30:20.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.337Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T07:30:20.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.337Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T07:30:20.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.338Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T07:30:20.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.338Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T07:30:20.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.340Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T07:30:20.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.341Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.664µs 2024-09-11T07:30:20.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.341Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T07:30:20.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.341Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-11T07:30:20.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.341Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=17.399µs wal_replay_duration=471.172µs wbl_replay_duration=111ns total_replay_duration=507.038µs 2024-09-11T07:30:20.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.342Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T07:30:20.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.342Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T07:30:20.363 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.342Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T07:30:20.363 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.358Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=16.179059ms db_storage=693ns remote_storage=1.035µs web_handler=336ns query_engine=503ns scrape=394.486µs scrape_sd=18.239µs notify=743ns notify_sd=1.302µs rules=15.575469ms tracing=6.209µs 2024-09-11T07:30:20.363 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.358Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T07:30:20.363 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:30:20.358Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T07:30:20.669 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 bash[64832]: 55c09ced2ef69c97dd9ead9f397957cca3a365e6f8f301603a4c81d7ae8a9a1d 2024-09-11T07:30:20.669 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:30:20 smithi164 systemd[1]: Started Ceph prometheus.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:30:21.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-mon[29604]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:30:21.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:21.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:21.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:21.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T07:30:21.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-mon[38451]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:30:21.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:21.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:21.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:21.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T07:30:21.420 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:21 smithi164 ceph-mon[33167]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:30:21.420 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:21 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:21.420 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:21 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:21.420 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:21 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' 2024-09-11T07:30:21.420 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:21 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T07:30:21.421 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:21 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:30:21.655 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ignoring --setuser ceph since I am not root 2024-09-11T07:30:21.655 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ignoring --setgroup ceph since I am not root 2024-09-11T07:30:21.655 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-mgr[29878]: -- 172.21.15.143:0/338064297 <== mon.2 v2:172.21.15.143:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55994e1fa340 con 0x55994e207000 2024-09-11T07:30:21.655 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:21.599+0000 7ff162495ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:30:21.655 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:21.653+0000 7ff162495ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:30:21.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:21 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ignoring --setuser ceph since I am not root 2024-09-11T07:30:21.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:21 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ignoring --setgroup ceph since I am not root 2024-09-11T07:30:21.717 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:21 smithi164 ceph-mgr[35255]: -- 172.21.15.164:0/3254501392 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562fa90e4340 con 0x562fa90f0800 2024-09-11T07:30:21.717 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:21 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:21.597+0000 7f8c05452ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:30:21.717 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:21 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:21.651+0000 7f8c05452ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:30:21.978 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:21 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:21.717+0000 7f8c05452ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:30:21.978 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:21 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:21.840+0000 7f8c05452ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:30:21.989 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:21.721+0000 7ff162495ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:30:21.990 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:21.848+0000 7ff162495ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:30:21.990 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:21 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:21.987+0000 7ff162495ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:30:22.210 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:30:22.235 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:21 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:21.976+0000 7f8c05452ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:30:22.236 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:22 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:22.236 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:22 smithi164 ceph-mon[33167]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T07:30:22.236 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:22 smithi164 ceph-mon[33167]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T07:30:22.269 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:22 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:22.269 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:22 smithi143 ceph-mon[29604]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T07:30:22.269 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:22 smithi143 ceph-mon[29604]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T07:30:22.269 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:22 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:22.269 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:22 smithi143 ceph-mon[38451]: from='mgr.14150 172.21.15.143:0/569552482' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T07:30:22.270 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:22 smithi143 ceph-mon[38451]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T07:30:22.543 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:22.267+0000 7ff162495ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:30:22.543 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:22.327+0000 7ff162495ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:30:22.544 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:22.492+0000 7ff162495ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:30:22.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:22 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:22.233+0000 7f8c05452ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:30:22.717 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:22 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:22.293+0000 7f8c05452ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:30:22.717 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:22 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:22.453+0000 7f8c05452ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:30:22.861 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2109174161}]},"active_addr":"172.21.15.143:6800/2109174161","active_change":"2024-09-11T07:25:37.628408+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24106,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.143:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":1036694372}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":1882273098}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":3135143747}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":2593822650}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":901994878}]}]} 2024-09-11T07:30:22.865 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T07:30:22.865 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T07:30:22.865 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T07:30:23.116 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:23.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:23 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/4156863257' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T07:30:23.466 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:23 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:23.292+0000 7f8c05452ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:30:23.467 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:23 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:23.355+0000 7f8c05452ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:30:23.497 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:23 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/4156863257' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T07:30:23.497 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:23 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:23.380+0000 7ff162495ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:30:23.497 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:23 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:23.448+0000 7ff162495ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:30:23.498 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:23 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/4156863257' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T07:30:24.226 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:24 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:24.160+0000 7ff162495ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:30:24.345 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:24.054+0000 7f8c05452ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:30:24.346 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:24.164+0000 7f8c05452ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:30:24.346 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:24.284+0000 7f8c05452ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:30:24.535 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:24 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:24.275+0000 7ff162495ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:30:24.535 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:24 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:24.401+0000 7ff162495ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:30:24.535 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:24 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:24.461+0000 7ff162495ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:30:24.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:24.343+0000 7f8c05452ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:30:24.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:24.468+0000 7f8c05452ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:30:24.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:24.534+0000 7f8c05452ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:30:24.717 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:24.591+0000 7f8c05452ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:30:24.798 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:24 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:24.593+0000 7ff162495ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:30:24.798 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:24 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:24.659+0000 7ff162495ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:30:24.799 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:24 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:24.720+0000 7ff162495ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:30:24.923 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:30:24.923 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":61,"fsid":"d3dfd926-700e-11ef-bcea-c7b262605968","created":"2024-09-11T07:24:43.922426+0000","modified":"2024-09-11T07:29:59.182442+0000","last_up_change":"2024-09-11T07:29:36.902426+0000","last_in_change":"2024-09-11T07:29:20.147581+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:27:45.934203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-11T07:29:47.688717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-11T07:29:47.920540+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T07:29:49.498899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T07:29:51.469678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T07:29:53.539340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"aafa4449-5343-4ce4-bb8a-1935f5e4fcbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6801","nonce":168444810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":168444810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":168444810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":168444810}]},"public_addr":"172.21.15.143:6801/168444810","cluster_addr":"172.21.15.143:6802/168444810","heartbeat_back_addr":"172.21.15.143:6804/168444810","heartbeat_front_addr":"172.21.15.143:6803/168444810","state":["exists","up"]},{"osd":1,"uuid":"633a8442-c551-4678-bd79-d938e83c11cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6805","nonce":507789371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":507789371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":507789371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":507789371}]},"public_addr":"172.21.15.143:6805/507789371","cluster_addr":"172.21.15.143:6806/507789371","heartbeat_back_addr":"172.21.15.143:6808/507789371","heartbeat_front_addr":"172.21.15.143:6807/507789371","state":["exists","up"]},{"osd":2,"uuid":"0cad9b00-6cc8-4cde-abe0-64f80bccb80b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6809","nonce":2660767941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2660767941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":2660767941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2660767941}]},"public_addr":"172.21.15.143:6809/2660767941","cluster_addr":"172.21.15.143:6810/2660767941","heartbeat_back_addr":"172.21.15.143:6812/2660767941","heartbeat_front_addr":"172.21.15.143:6811/2660767941","state":["exists","up"]},{"osd":3,"uuid":"b4a7c5e6-6afd-4bfc-8db9-ee6c4349061f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":2376878046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2376878046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2376878046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6815","nonce":2376878046}]},"public_addr":"172.21.15.143:6813/2376878046","cluster_addr":"172.21.15.143:6814/2376878046","heartbeat_back_addr":"172.21.15.143:6816/2376878046","heartbeat_front_addr":"172.21.15.143:6815/2376878046","state":["exists","up"]},{"osd":4,"uuid":"61aa7852-9e2a-4498-9f68-b248012431a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2286891940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6801","nonce":2286891940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6803","nonce":2286891940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2286891940}]},"public_addr":"172.21.15.164:6800/2286891940","cluster_addr":"172.21.15.164:6801/2286891940","heartbeat_back_addr":"172.21.15.164:6803/2286891940","heartbeat_front_addr":"172.21.15.164:6802/2286891940","state":["exists","up"]},{"osd":5,"uuid":"bd3428b7-b0c6-474e-b462-d964c1b91327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4094877797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6805","nonce":4094877797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6807","nonce":4094877797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4094877797}]},"public_addr":"172.21.15.164:6804/4094877797","cluster_addr":"172.21.15.164:6805/4094877797","heartbeat_back_addr":"172.21.15.164:6807/4094877797","heartbeat_front_addr":"172.21.15.164:6806/4094877797","state":["exists","up"]},{"osd":6,"uuid":"abbb121a-3d68-4661-8087-f8630232871e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":174920052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":174920052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6811","nonce":174920052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":174920052}]},"public_addr":"172.21.15.164:6808/174920052","cluster_addr":"172.21.15.164:6809/174920052","heartbeat_back_addr":"172.21.15.164:6811/174920052","heartbeat_front_addr":"172.21.15.164:6810/174920052","state":["exists","up"]},{"osd":7,"uuid":"f3b1d8ce-1a3a-4da5-bcf2-46d1aeb38b95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1031390908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":1031390908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6815","nonce":1031390908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1031390908}]},"public_addr":"172.21.15.164:6812/1031390908","cluster_addr":"172.21.15.164:6813/1031390908","heartbeat_back_addr":"172.21.15.164:6815/1031390908","heartbeat_front_addr":"172.21.15.164:6814/1031390908","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:56.642492+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:27:19.610748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:27:42.158678+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:28:05.261976+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:28:26.646599+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:28:49.900951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:29:12.845103+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:29:34.961584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/4260814966":"2024-09-12T07:25:37.628302+0000","172.21.15.143:0/3068436287":"2024-09-12T07:25:37.628302+0000","172.21.15.143:0/2077730301":"2024-09-12T07:25:15.340448+0000","172.21.15.143:0/105787121":"2024-09-12T07:25:15.340448+0000","172.21.15.143:0/1457644251":"2024-09-12T07:25:15.340448+0000","172.21.15.143:0/2914000301":"2024-09-12T07:25:37.628302+0000","172.21.15.143:6800/2276713308":"2024-09-12T07:25:37.628302+0000","172.21.15.143:6800/1120431981":"2024-09-12T07:25:15.340448+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T07:30:24.993 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:24 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2304324939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:30:24.994 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:24.725+0000 7f8c05452ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:30:24.994 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:24.788+0000 7f8c05452ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:30:24.994 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:24.852+0000 7f8c05452ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:30:24.994 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:24.932+0000 7f8c05452ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:30:24.994 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:24.990+0000 7f8c05452ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:30:25.136 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:24 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2304324939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:30:25.136 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:24 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2304324939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:30:25.136 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:24 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:24.861+0000 7ff162495ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:30:25.136 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:24 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:24.926+0000 7ff162495ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:30:25.136 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:24 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:24.995+0000 7ff162495ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:30:25.136 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:25 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:25.076+0000 7ff162495ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:30:25.136 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:25 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:25.133+0000 7ff162495ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:30:25.466 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:25.048+0000 7f8c05452ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:30:25.548 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:25 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:25.192+0000 7ff162495ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:30:25.651 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T07:30:25.651 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T07:30:25.907 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:26.183 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:25 smithi164 ceph-mon[33167]: Standby manager daemon x restarted 2024-09-11T07:30:26.183 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:25 smithi164 ceph-mon[33167]: Standby manager daemon x started 2024-09-11T07:30:26.184 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:25 smithi164 ceph-mon[33167]: from='mgr.? 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:30:26.184 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:25 smithi164 ceph-mon[33167]: from='mgr.? 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:30:26.184 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:25 smithi164 ceph-mon[33167]: from='mgr.? 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:30:26.184 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:25 smithi164 ceph-mon[33167]: from='mgr.? 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:30:26.184 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:25.795+0000 7f8c05452ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:30:26.184 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: [11/Sep/2024:07:30:25] ENGINE Bus STARTING 2024-09-11T07:30:26.184 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: CherryPy Checker: 2024-09-11T07:30:26.184 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: The Application mounted at '' has an empty config. 2024-09-11T07:30:26.184 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:30:26.184 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: [11/Sep/2024:07:30:25] ENGINE Serving on http://:::9283 2024-09-11T07:30:26.185 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: [11/Sep/2024:07:30:25] ENGINE Bus STARTED 2024-09-11T07:30:26.297 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:25 smithi143 ceph-mon[29604]: Standby manager daemon x restarted 2024-09-11T07:30:26.297 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:25 smithi143 ceph-mon[29604]: Standby manager daemon x started 2024-09-11T07:30:26.297 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:25 smithi143 ceph-mon[29604]: from='mgr.? 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:30:26.297 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:25 smithi143 ceph-mon[29604]: from='mgr.? 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:30:26.297 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:25 smithi143 ceph-mon[29604]: from='mgr.? 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:30:26.297 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:25 smithi143 ceph-mon[29604]: from='mgr.? 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:30:26.298 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:25 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:25.973+0000 7ff162495ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:30:26.298 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:30:26] ENGINE Bus STARTING 2024-09-11T07:30:26.298 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: CherryPy Checker: 2024-09-11T07:30:26.298 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: The Application mounted at '' has an empty config. 2024-09-11T07:30:26.298 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:26.298 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:30:26] ENGINE Serving on http://:::9283 2024-09-11T07:30:26.298 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:30:26] ENGINE Bus STARTED 2024-09-11T07:30:26.298 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:25 smithi143 ceph-mon[38451]: Standby manager daemon x restarted 2024-09-11T07:30:26.298 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:25 smithi143 ceph-mon[38451]: Standby manager daemon x started 2024-09-11T07:30:26.298 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:25 smithi143 ceph-mon[38451]: from='mgr.? 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:30:26.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:25 smithi143 ceph-mon[38451]: from='mgr.? 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:30:26.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:25 smithi143 ceph-mon[38451]: from='mgr.? 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:30:26.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:25 smithi143 ceph-mon[38451]: from='mgr.? 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:30:27.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: Active manager daemon y restarted 2024-09-11T07:30:27.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: Activating manager daemon y 2024-09-11T07:30:27.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-11T07:30:27.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T07:30:27.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: mgrmap e19: y(active, starting, since 0.00732649s), standbys: x 2024-09-11T07:30:27.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:30:27.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:30:27.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:30:27.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:30:27.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:30:27.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:30:27.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:30:27.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:30:27.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:30:27.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:30:27.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:30:27.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:30:27.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:30:27.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:30:27.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:30:27.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:30:27.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: Manager daemon y is now available 2024-09-11T07:30:27.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:30:27.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:27.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:27.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:30:27.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:30:27.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: Active manager daemon y restarted 2024-09-11T07:30:27.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: Activating manager daemon y 2024-09-11T07:30:27.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-11T07:30:27.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T07:30:27.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: mgrmap e19: y(active, starting, since 0.00732649s), standbys: x 2024-09-11T07:30:27.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:30:27.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:30:27.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:30:27.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:30:27.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:30:27.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:30:27.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:30:27.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:30:27.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:30:27.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:30:27.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:30:27.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:30:27.303 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:30:27.303 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:30:27.303 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:30:27.303 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:30:27.303 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: Manager daemon y is now available 2024-09-11T07:30:27.303 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:30:27.303 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:27.303 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:27.303 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:30:27.303 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:30:27.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: Active manager daemon y restarted 2024-09-11T07:30:27.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: Activating manager daemon y 2024-09-11T07:30:27.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-11T07:30:27.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T07:30:27.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: mgrmap e19: y(active, starting, since 0.00732649s), standbys: x 2024-09-11T07:30:27.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:30:27.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:30:27.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:30:27.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:30:27.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:30:27.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:30:27.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:30:27.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:30:27.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:30:27.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:30:27.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:30:27.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:30:27.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:30:27.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:30:27.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:30:27.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:30:27.469 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: Manager daemon y is now available 2024-09-11T07:30:27.469 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:30:27.469 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:27.469 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:27.469 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:30:27.469 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:30:27.772 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:27.367+0000 7ff105217640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:07:30:27] ENGINE Error in HTTPServer.serve 2024-09-11T07:30:27.773 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: Traceback (most recent call last): 2024-09-11T07:30:27.773 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:30:27.773 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: self._connections.run(self.expiration_interval) 2024-09-11T07:30:27.773 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:30:27.773 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: self._run(expiration_interval) 2024-09-11T07:30:27.773 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:30:27.773 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:30:27.773 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:30:27.773 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:30:27.773 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:30:27.773 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: s = self.context.wrap_socket( 2024-09-11T07:30:27.773 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:30:27.773 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: return self.sslsocket_class._create( 2024-09-11T07:30:27.774 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:30:27.774 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: self.do_handshake() 2024-09-11T07:30:27.774 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:30:27.774 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: self._sslobj.do_handshake() 2024-09-11T07:30:27.774 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:30:27.774 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:30:28.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-mon[29604]: mgrmap e20: y(active, since 1.00902s), standbys: x 2024-09-11T07:30:28.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-mon[29604]: [11/Sep/2024:07:30:27] ENGINE Bus STARTING 2024-09-11T07:30:28.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-mon[29604]: [11/Sep/2024:07:30:27] ENGINE Serving on https://172.21.15.143:7150 2024-09-11T07:30:28.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-mon[29604]: [11/Sep/2024:07:30:27] ENGINE Bus STARTED 2024-09-11T07:30:28.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-mon[29604]: [11/Sep/2024:07:30:27] ENGINE Error in HTTPServer.serve 2024-09-11T07:30:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout: Traceback (most recent call last): 2024-09-11T07:30:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:30:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:30:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:30:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._run(expiration_interval) 2024-09-11T07:30:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:30:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:30:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:30:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:30:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:30:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout: s = self.context.wrap_socket( 2024-09-11T07:30:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:30:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout: return self.sslsocket_class._create( 2024-09-11T07:30:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:30:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout: self.do_handshake() 2024-09-11T07:30:28.051 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:30:28.051 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._sslobj.do_handshake() 2024-09-11T07:30:28.051 INFO:journalctl@ceph.mon.a.smithi143.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:30:28.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:28.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-mon[38451]: mgrmap e20: y(active, since 1.00902s), standbys: x 2024-09-11T07:30:28.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-mon[38451]: [11/Sep/2024:07:30:27] ENGINE Bus STARTING 2024-09-11T07:30:28.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-mon[38451]: [11/Sep/2024:07:30:27] ENGINE Serving on https://172.21.15.143:7150 2024-09-11T07:30:28.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-mon[38451]: [11/Sep/2024:07:30:27] ENGINE Bus STARTED 2024-09-11T07:30:28.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-mon[38451]: [11/Sep/2024:07:30:27] ENGINE Error in HTTPServer.serve 2024-09-11T07:30:28.051 INFO:journalctl@ceph.mon.c.smithi143.stdout: Traceback (most recent call last): 2024-09-11T07:30:28.051 INFO:journalctl@ceph.mon.c.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:30:28.052 INFO:journalctl@ceph.mon.c.smithi143.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:30:28.052 INFO:journalctl@ceph.mon.c.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:30:28.052 INFO:journalctl@ceph.mon.c.smithi143.stdout: self._run(expiration_interval) 2024-09-11T07:30:28.052 INFO:journalctl@ceph.mon.c.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:30:28.052 INFO:journalctl@ceph.mon.c.smithi143.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:30:28.052 INFO:journalctl@ceph.mon.c.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:30:28.052 INFO:journalctl@ceph.mon.c.smithi143.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:30:28.052 INFO:journalctl@ceph.mon.c.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:30:28.052 INFO:journalctl@ceph.mon.c.smithi143.stdout: s = self.context.wrap_socket( 2024-09-11T07:30:28.052 INFO:journalctl@ceph.mon.c.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:30:28.052 INFO:journalctl@ceph.mon.c.smithi143.stdout: return self.sslsocket_class._create( 2024-09-11T07:30:28.052 INFO:journalctl@ceph.mon.c.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:30:28.052 INFO:journalctl@ceph.mon.c.smithi143.stdout: self.do_handshake() 2024-09-11T07:30:28.052 INFO:journalctl@ceph.mon.c.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:30:28.053 INFO:journalctl@ceph.mon.c.smithi143.stdout: self._sslobj.do_handshake() 2024-09-11T07:30:28.053 INFO:journalctl@ceph.mon.c.smithi143.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:30:28.053 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:27 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:28.416 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:30:28.416 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":62,"fsid":"d3dfd926-700e-11ef-bcea-c7b262605968","created":"2024-09-11T07:24:43.922426+0000","modified":"2024-09-11T07:30:25.981507+0000","last_up_change":"2024-09-11T07:29:36.902426+0000","last_in_change":"2024-09-11T07:29:20.147581+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:27:45.934203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-11T07:29:47.688717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-11T07:29:47.920540+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T07:29:49.498899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T07:29:51.469678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T07:29:53.539340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"aafa4449-5343-4ce4-bb8a-1935f5e4fcbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6801","nonce":168444810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":168444810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":168444810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":168444810}]},"public_addr":"172.21.15.143:6801/168444810","cluster_addr":"172.21.15.143:6802/168444810","heartbeat_back_addr":"172.21.15.143:6804/168444810","heartbeat_front_addr":"172.21.15.143:6803/168444810","state":["exists","up"]},{"osd":1,"uuid":"633a8442-c551-4678-bd79-d938e83c11cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6805","nonce":507789371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":507789371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":507789371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":507789371}]},"public_addr":"172.21.15.143:6805/507789371","cluster_addr":"172.21.15.143:6806/507789371","heartbeat_back_addr":"172.21.15.143:6808/507789371","heartbeat_front_addr":"172.21.15.143:6807/507789371","state":["exists","up"]},{"osd":2,"uuid":"0cad9b00-6cc8-4cde-abe0-64f80bccb80b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6809","nonce":2660767941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2660767941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":2660767941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2660767941}]},"public_addr":"172.21.15.143:6809/2660767941","cluster_addr":"172.21.15.143:6810/2660767941","heartbeat_back_addr":"172.21.15.143:6812/2660767941","heartbeat_front_addr":"172.21.15.143:6811/2660767941","state":["exists","up"]},{"osd":3,"uuid":"b4a7c5e6-6afd-4bfc-8db9-ee6c4349061f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6813","nonce":2376878046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2376878046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2376878046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6815","nonce":2376878046}]},"public_addr":"172.21.15.143:6813/2376878046","cluster_addr":"172.21.15.143:6814/2376878046","heartbeat_back_addr":"172.21.15.143:6816/2376878046","heartbeat_front_addr":"172.21.15.143:6815/2376878046","state":["exists","up"]},{"osd":4,"uuid":"61aa7852-9e2a-4498-9f68-b248012431a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2286891940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6801","nonce":2286891940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6803","nonce":2286891940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2286891940}]},"public_addr":"172.21.15.164:6800/2286891940","cluster_addr":"172.21.15.164:6801/2286891940","heartbeat_back_addr":"172.21.15.164:6803/2286891940","heartbeat_front_addr":"172.21.15.164:6802/2286891940","state":["exists","up"]},{"osd":5,"uuid":"bd3428b7-b0c6-474e-b462-d964c1b91327","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4094877797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6805","nonce":4094877797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6807","nonce":4094877797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4094877797}]},"public_addr":"172.21.15.164:6804/4094877797","cluster_addr":"172.21.15.164:6805/4094877797","heartbeat_back_addr":"172.21.15.164:6807/4094877797","heartbeat_front_addr":"172.21.15.164:6806/4094877797","state":["exists","up"]},{"osd":6,"uuid":"abbb121a-3d68-4661-8087-f8630232871e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":174920052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":174920052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6811","nonce":174920052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":174920052}]},"public_addr":"172.21.15.164:6808/174920052","cluster_addr":"172.21.15.164:6809/174920052","heartbeat_back_addr":"172.21.15.164:6811/174920052","heartbeat_front_addr":"172.21.15.164:6810/174920052","state":["exists","up"]},{"osd":7,"uuid":"f3b1d8ce-1a3a-4da5-bcf2-46d1aeb38b95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1031390908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":1031390908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6815","nonce":1031390908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1031390908}]},"public_addr":"172.21.15.164:6812/1031390908","cluster_addr":"172.21.15.164:6813/1031390908","heartbeat_back_addr":"172.21.15.164:6815/1031390908","heartbeat_front_addr":"172.21.15.164:6814/1031390908","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:56.642492+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:27:19.610748+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:27:42.158678+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:28:05.261976+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:28:26.646599+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:28:49.900951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:29:12.845103+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:29:34.961584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/2593822650":"2024-09-12T07:30:25.981461+0000","172.21.15.143:6800/1120431981":"2024-09-12T07:25:15.340448+0000","172.21.15.143:0/1036694372":"2024-09-12T07:30:25.981461+0000","172.21.15.143:0/901994878":"2024-09-12T07:30:25.981461+0000","172.21.15.143:0/1457644251":"2024-09-12T07:25:15.340448+0000","172.21.15.143:0/105787121":"2024-09-12T07:25:15.340448+0000","172.21.15.143:0/2077730301":"2024-09-12T07:25:15.340448+0000","172.21.15.143:6800/2109174161":"2024-09-12T07:30:25.981461+0000","172.21.15.143:0/2914000301":"2024-09-12T07:25:37.628302+0000","172.21.15.143:0/3068436287":"2024-09-12T07:25:37.628302+0000","172.21.15.143:0/4260814966":"2024-09-12T07:25:37.628302+0000","172.21.15.143:6800/2276713308":"2024-09-12T07:25:37.628302+0000","172.21.15.143:0/3135143747":"2024-09-12T07:30:25.981461+0000","172.21.15.143:0/1882273098":"2024-09-12T07:30:25.981461+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T07:30:28.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:27 smithi164 ceph-mon[33167]: mgrmap e20: y(active, since 1.00902s), standbys: x 2024-09-11T07:30:28.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:27 smithi164 ceph-mon[33167]: [11/Sep/2024:07:30:27] ENGINE Bus STARTING 2024-09-11T07:30:28.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:27 smithi164 ceph-mon[33167]: [11/Sep/2024:07:30:27] ENGINE Serving on https://172.21.15.143:7150 2024-09-11T07:30:28.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:27 smithi164 ceph-mon[33167]: [11/Sep/2024:07:30:27] ENGINE Bus STARTED 2024-09-11T07:30:28.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:27 smithi164 ceph-mon[33167]: [11/Sep/2024:07:30:27] ENGINE Error in HTTPServer.serve 2024-09-11T07:30:28.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: Traceback (most recent call last): 2024-09-11T07:30:28.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:30:28.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:30:28.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:30:28.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: self._run(expiration_interval) 2024-09-11T07:30:28.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:30:28.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:30:28.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:30:28.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:30:28.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:30:28.468 INFO:journalctl@ceph.mon.b.smithi164.stdout: s = self.context.wrap_socket( 2024-09-11T07:30:28.468 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:30:28.468 INFO:journalctl@ceph.mon.b.smithi164.stdout: return self.sslsocket_class._create( 2024-09-11T07:30:28.468 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:30:28.468 INFO:journalctl@ceph.mon.b.smithi164.stdout: self.do_handshake() 2024-09-11T07:30:28.468 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:30:28.468 INFO:journalctl@ceph.mon.b.smithi164.stdout: self._sslobj.do_handshake() 2024-09-11T07:30:28.468 INFO:journalctl@ceph.mon.b.smithi164.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:30:28.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:27 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:29.373 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T07:30:29.374 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T07:30:29.374 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T07:30:29.375 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T07:30:29.375 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T07:30:29.375 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T07:30:29.376 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T07:30:29.376 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T07:30:29.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:29 smithi164 ceph-mon[33167]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:30:29.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:29 smithi164 ceph-mon[33167]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-11T07:30:29.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:29 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1220980123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:30:29.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:29 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:29.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:29 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:29.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:29 smithi143 ceph-mon[29604]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:30:29.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:29 smithi143 ceph-mon[29604]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-11T07:30:29.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:29 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1220980123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:30:29.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:29 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:29.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:29 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:29.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:29 smithi143 ceph-mon[38451]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:30:29.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:29 smithi143 ceph-mon[38451]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-11T07:30:29.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:29 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1220980123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:30:29.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:29 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:29.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:29 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:29.885 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:29.893 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:29.893 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:29.899 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:29.903 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:29.903 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:30.386 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:30.387 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:31.008 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:31.008 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:31.008 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:31.008 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:31.008 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:31.008 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:31.008 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[29604]: Adjusting osd_memory_target on smithi164 to 3760M 2024-09-11T07:30:31.008 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:31.008 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[29604]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:30:31.009 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:31.009 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:31.009 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:31.009 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:31.009 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:31.009 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:31.009 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[38451]: Adjusting osd_memory_target on smithi164 to 3760M 2024-09-11T07:30:31.009 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:31.009 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:30 smithi143 ceph-mon[38451]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:30:31.178 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:30 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:31.178 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:30 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:31.178 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:30 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:31.178 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:30 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:31.178 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:30 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:31.178 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:30 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:31.179 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:30 smithi164 ceph-mon[33167]: Adjusting osd_memory_target on smithi164 to 3760M 2024-09-11T07:30:31.179 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:30 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:31.179 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:30 smithi164 ceph-mon[33167]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:30:31.466 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:31 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:30:31.942 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:31 smithi143 ceph-mon[29604]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-11T07:30:31.942 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:31 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:31.942 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:31 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:31.942 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:31 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:31.942 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:31 smithi143 ceph-mon[38451]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-11T07:30:31.943 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:31 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:31.943 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:31 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:31.943 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:31 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:32.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:31 smithi164 ceph-mon[33167]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-11T07:30:32.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:31 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:32.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:31 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:32.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:31 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:33.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:32 smithi143 ceph-mon[29604]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1 KiB/s rd, 1 op/s 2024-09-11T07:30:33.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:32 smithi143 ceph-mon[38451]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1 KiB/s rd, 1 op/s 2024-09-11T07:30:33.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:32 smithi164 ceph-mon[33167]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1 KiB/s rd, 1 op/s 2024-09-11T07:30:34.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:30:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:30:35.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:35 smithi164 ceph-mon[33167]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T07:30:35.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:35 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:35.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:35 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:35.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:35 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:35.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:35 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:35.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:35 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:35.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:35 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:35.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:35 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:35.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:35 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:30:35.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:35 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:30:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[29604]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T07:30:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:30:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:30:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[38451]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T07:30:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:30:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:30:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:35 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:30:36.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[38451]: Adjusting osd_memory_target on smithi143 to 3760M 2024-09-11T07:30:36.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[38451]: Updating smithi143:/etc/ceph/ceph.conf 2024-09-11T07:30:36.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[38451]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-11T07:30:36.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[38451]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:30:36.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[38451]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:30:36.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[38451]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:30:36.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[38451]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:30:36.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[38451]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:30:36.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[38451]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:30:36.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:36.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:36.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:36.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:36.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:36.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[38451]: Deploying daemon alertmanager.a on smithi143 2024-09-11T07:30:36.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[29604]: Adjusting osd_memory_target on smithi143 to 3760M 2024-09-11T07:30:36.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[29604]: Updating smithi143:/etc/ceph/ceph.conf 2024-09-11T07:30:36.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[29604]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-11T07:30:36.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[29604]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:30:36.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[29604]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:30:36.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[29604]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:30:36.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[29604]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:30:36.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[29604]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:30:36.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[29604]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:30:36.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:36.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:36.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:36.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:36.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:36.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:36 smithi143 ceph-mon[29604]: Deploying daemon alertmanager.a on smithi143 2024-09-11T07:30:36.629 INFO:teuthology.orchestra.run.smithi143.stdout:158913789974 2024-09-11T07:30:36.630 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T07:30:36.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:36 smithi164 ceph-mon[33167]: Adjusting osd_memory_target on smithi143 to 3760M 2024-09-11T07:30:36.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:36 smithi164 ceph-mon[33167]: Updating smithi143:/etc/ceph/ceph.conf 2024-09-11T07:30:36.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:36 smithi164 ceph-mon[33167]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-11T07:30:36.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:36 smithi164 ceph-mon[33167]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:30:36.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:36 smithi164 ceph-mon[33167]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:30:36.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:36 smithi164 ceph-mon[33167]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:30:36.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:36 smithi164 ceph-mon[33167]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:30:36.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:36 smithi164 ceph-mon[33167]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:30:36.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:36 smithi164 ceph-mon[33167]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:30:36.718 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:36 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:36.718 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:36 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:36.718 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:36 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:36.718 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:36 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:36.718 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:36 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:30:36.718 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:36 smithi164 ceph-mon[33167]: Deploying daemon alertmanager.a on smithi143 2024-09-11T07:30:37.611 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:37 smithi143 ceph-mon[29604]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T07:30:37.611 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:37 smithi143 ceph-mon[38451]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T07:30:37.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:37 smithi164 ceph-mon[33167]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T07:30:39.603 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:39 smithi143 ceph-mon[29604]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:30:39.604 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:39 smithi143 ceph-mon[38451]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:30:39.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:39 smithi164 ceph-mon[33167]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:30:41.466 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:30:41.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:41 smithi143 ceph-mon[29604]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:30:41.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:41 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:41.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:41 smithi143 ceph-mon[38451]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:30:41.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:41 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:41.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:41 smithi164 ceph-mon[33167]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:30:41.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:41 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:42.182 INFO:teuthology.orchestra.run.smithi143.stdout:111669149729 2024-09-11T07:30:42.182 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T07:30:42.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:42 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:42.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:42 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:42.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:42 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:43.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:43 smithi143 ceph-mon[29604]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:30:43.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:43 smithi143 ceph-mon[38451]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:30:43.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:43 smithi164 ceph-mon[33167]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:30:44.487 INFO:teuthology.orchestra.run.smithi143.stdout:210453397518 2024-09-11T07:30:44.487 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T07:30:44.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:44 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:30:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:30:45.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:45 smithi143 ceph-mon[29604]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:30:45.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:45 smithi143 ceph-mon[38451]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:30:45.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:45 smithi164 ceph-mon[33167]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:30:46.460 INFO:teuthology.orchestra.run.smithi143.stdout:55834574891 2024-09-11T07:30:46.461 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T07:30:47.292 INFO:teuthology.orchestra.run.smithi143.stdout:77309411365 2024-09-11T07:30:47.293 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T07:30:47.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:47 smithi143 ceph-mon[29604]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:30:47.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:47 smithi143 ceph-mon[38451]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:30:47.727 INFO:teuthology.orchestra.run.smithi143.stdout:34359738415 2024-09-11T07:30:47.727 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T07:30:47.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:47 smithi164 ceph-mon[33167]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:30:49.769 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:49 smithi164 ceph-mon[33167]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:30:49.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:49 smithi143 ceph-mon[29604]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:30:49.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:49 smithi143 ceph-mon[38451]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:30:51.309 INFO:teuthology.orchestra.run.smithi143.stdout:133143986206 2024-09-11T07:30:51.309 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T07:30:51.431 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:51.431 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:51.434 INFO:teuthology.orchestra.run.smithi143.stdout:184683593748 2024-09-11T07:30:51.434 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T07:30:51.436 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:51.438 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:51.466 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:30:51 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:30:51.533 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:51 smithi143 ceph-mon[29604]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:30:51.534 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:51 smithi143 ceph-mon[38451]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:30:51.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:51 smithi164 ceph-mon[33167]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:30:51.971 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:52.683 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:52 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:52.683 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:52 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:52.872 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:52.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:52 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:53.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:53 smithi143 ceph-mon[29604]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:30:53.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:53 smithi143 ceph-mon[38451]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:30:53.850 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:53.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:53 smithi164 ceph-mon[33167]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:30:54.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:30:54 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:30:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:30:55.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:55 smithi143 ceph-mon[29604]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:30:55.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:55 smithi143 ceph-mon[38451]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:30:55.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:55 smithi164 ceph-mon[33167]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:30:56.299 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:30:56 smithi143 podman[72681]: 2024-09-11 07:30:56.238974888 +0000 UTC m=+2.003057286 container create 2ef0e3968f2476da7241bd64e63acbe8abefcf13677beb65258e07a60f19054d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:30:56.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:56 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:56.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:56 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:56.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:56 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:57.619 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:57 smithi143 ceph-mon[29604]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T07:30:57.619 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:57 smithi143 ceph-mon[38451]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T07:30:57.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:57 smithi164 ceph-mon[33167]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T07:30:58.866 INFO:teuthology.orchestra.run.smithi143.stdout:55834574894 2024-09-11T07:30:59.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:59 smithi143 ceph-mon[29604]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:30:59.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:30:59 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3910248604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T07:30:59.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:59 smithi143 ceph-mon[38451]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:30:59.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:30:59 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3910248604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T07:30:59.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:59 smithi164 ceph-mon[33167]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:30:59.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:30:59 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3910248604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T07:31:01.466 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:31:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:31:01.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:01 smithi143 ceph-mon[29604]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:01.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:01 smithi143 ceph-mon[38451]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:01.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:01 smithi164 ceph-mon[33167]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:02.646 INFO:teuthology.orchestra.run.smithi143.stdout:210453397523 2024-09-11T07:31:02.646 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:02 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:02.646 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:02 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1442194437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T07:31:02.647 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:02 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:02.647 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:02 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1442194437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T07:31:02.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:02 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:02.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:02 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1442194437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T07:31:03.400 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397518 got 210453397523 for osd.7 2024-09-11T07:31:03.401 DEBUG:teuthology.parallel:result is None 2024-09-11T07:31:03.567 INFO:teuthology.orchestra.run.smithi143.stdout:111669149733 2024-09-11T07:31:03.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:03 smithi143 ceph-mon[29604]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:03.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:03 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3344589191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T07:31:03.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:03 smithi143 ceph-mon[38451]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:03.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:03 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3344589191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T07:31:03.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:03 smithi164 ceph-mon[33167]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:03.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:03 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3344589191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T07:31:04.231 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:03 smithi143 podman[72681]: 2024-09-11 07:31:03.844323839 +0000 UTC m=+9.608406250 container init 2ef0e3968f2476da7241bd64e63acbe8abefcf13677beb65258e07a60f19054d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:04.231 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:03 smithi143 podman[72681]: 2024-09-11 07:31:03.851622371 +0000 UTC m=+9.615704766 container start 2ef0e3968f2476da7241bd64e63acbe8abefcf13677beb65258e07a60f19054d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:04.232 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:03 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[73059]: ts=2024-09-11T07:31:03.866Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-11T07:31:04.232 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:03 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[73059]: ts=2024-09-11T07:31:03.866Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-11T07:31:04.232 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:03 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[73059]: ts=2024-09-11T07:31:03.867Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.143 port=9094 2024-09-11T07:31:04.232 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:03 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[73059]: ts=2024-09-11T07:31:03.868Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-11T07:31:04.232 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:03 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[73059]: ts=2024-09-11T07:31:03.893Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T07:31:04.232 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:03 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[73059]: ts=2024-09-11T07:31:03.893Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T07:31:04.232 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:03 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[73059]: ts=2024-09-11T07:31:03.896Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-11T07:31:04.232 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:03 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[73059]: ts=2024-09-11T07:31:03.896Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-11T07:31:04.261 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149729 got 111669149733 for osd.3 2024-09-11T07:31:04.261 DEBUG:teuthology.parallel:result is None 2024-09-11T07:31:04.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:04 smithi164 ceph-mon[33167]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:04.717 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:31:04 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:31:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:31:04.769 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:04 smithi143 ceph-mon[38451]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:04.769 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:04 smithi143 ceph-mon[29604]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:05.048 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:04 smithi143 bash[72681]: 2ef0e3968f2476da7241bd64e63acbe8abefcf13677beb65258e07a60f19054d 2024-09-11T07:31:05.049 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:04 smithi143 systemd[1]: Started Ceph alertmanager.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:31:05.326 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:05] ENGINE Bus STOPPING 2024-09-11T07:31:05.327 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:05] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:31:05.327 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:05] ENGINE Bus STOPPED 2024-09-11T07:31:05.327 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:05] ENGINE Bus STARTING 2024-09-11T07:31:05.327 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:05] ENGINE Serving on http://:::9283 2024-09-11T07:31:05.327 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:05] ENGINE Bus STARTED 2024-09-11T07:31:05.339 INFO:teuthology.orchestra.run.smithi143.stdout:158913789980 2024-09-11T07:31:06.048 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[73059]: ts=2024-09-11T07:31:05.869Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.00054132s 2024-09-11T07:31:06.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:31:06.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[29604]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:31:06.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[29604]: Deploying daemon grafana.a on smithi164 2024-09-11T07:31:06.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2402486620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T07:31:06.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2682398848' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T07:31:06.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:31:06.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[38451]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:31:06.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[38451]: Deploying daemon grafana.a on smithi164 2024-09-11T07:31:06.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2402486620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T07:31:06.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:05 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2682398848' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T07:31:06.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:05 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:05 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:05 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:05 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:05 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:05 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:05 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:31:06.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:05 smithi164 ceph-mon[33167]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:31:06.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:05 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:06.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:05 smithi164 ceph-mon[33167]: Deploying daemon grafana.a on smithi164 2024-09-11T07:31:06.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:05 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2402486620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T07:31:06.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:05 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2682398848' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T07:31:06.509 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574891 got 55834574894 for osd.1 2024-09-11T07:31:06.509 DEBUG:teuthology.parallel:result is None 2024-09-11T07:31:06.617 INFO:teuthology.orchestra.run.smithi143.stdout:34359738420 2024-09-11T07:31:07.172 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:07 smithi143 ceph-mon[38451]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:07.172 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:07 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:07.172 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:07 smithi143 ceph-mon[29604]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:07.172 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:07 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:07.203 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738415 got 34359738420 for osd.0 2024-09-11T07:31:07.203 DEBUG:teuthology.parallel:result is None 2024-09-11T07:31:07.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:07 smithi164 ceph-mon[33167]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:07.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:07 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:08.001 INFO:teuthology.orchestra.run.smithi143.stdout:77309411370 2024-09-11T07:31:08.179 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789974 got 158913789980 for osd.5 2024-09-11T07:31:08.180 DEBUG:teuthology.parallel:result is None 2024-09-11T07:31:08.264 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:08 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1940778196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T07:31:08.264 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:08 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1940778196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T07:31:08.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:08 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1940778196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T07:31:09.020 INFO:teuthology.orchestra.run.smithi143.stdout:184683593753 2024-09-11T07:31:09.082 INFO:teuthology.orchestra.run.smithi143.stdout:133143986210 2024-09-11T07:31:09.083 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:09 smithi143 ceph-mon[29604]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:09.083 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:09 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3014813633' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T07:31:09.083 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:09 smithi143 ceph-mon[38451]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:09.083 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:09 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3014813633' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T07:31:09.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:09 smithi164 ceph-mon[33167]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:09.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:09 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3014813633' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T07:31:09.836 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593748 got 184683593753 for osd.6 2024-09-11T07:31:09.837 DEBUG:teuthology.parallel:result is None 2024-09-11T07:31:10.089 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:10 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/4242285523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T07:31:10.090 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:10 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/4242285523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T07:31:10.392 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411365 got 77309411370 for osd.2 2024-09-11T07:31:10.393 DEBUG:teuthology.parallel:result is None 2024-09-11T07:31:10.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:10 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/4242285523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T07:31:11.039 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986206 got 133143986210 for osd.4 2024-09-11T07:31:11.039 DEBUG:teuthology.parallel:result is None 2024-09-11T07:31:11.039 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T07:31:11.040 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:31:11.295 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:11.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:11 smithi143 ceph-mon[29604]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:11.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:11 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:11.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:11 smithi143 ceph-mon[38451]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:11.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:11 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:11.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:11 smithi164 ceph-mon[33167]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:11.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:11 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:11.467 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:31:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:31:12.458 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:12 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:12.458 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:12 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:12.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:12 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:13.167 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:31:13.169 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-09-11T07:31:13.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:13 smithi164 ceph-mon[33167]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:13.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:13 smithi143 ceph-mon[29604]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:13.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:13 smithi143 ceph-mon[38451]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:13.866 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-11T07:31:12.005504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":929,"num_read_kb":786,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385440,"kb_used_data":6604,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575776,"statfs":{"total":767960285184,"available":765517594624,"internally_reserved":0,"allocated":6762496,"data_stored":3649554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002676"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310542+0000","last_change":"2024-09-11T07:29:55.331068+0000","last_active":"2024-09-11T07:29:59.310542+0000","last_peered":"2024-09-11T07:29:59.310542+0000","last_clean":"2024-09-11T07:29:59.310542+0000","last_became_active":"2024-09-11T07:29:55.330747+0000","last_became_peered":"2024-09-11T07:29:55.330747+0000","last_unstale":"2024-09-11T07:29:59.310542+0000","last_undegraded":"2024-09-11T07:29:59.310542+0000","last_fullsized":"2024-09-11T07:29:59.310542+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:12.694045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192388+0000","last_change":"2024-09-11T07:29:49.220298+0000","last_active":"2024-09-11T07:29:59.192388+0000","last_peered":"2024-09-11T07:29:59.192388+0000","last_clean":"2024-09-11T07:29:59.192388+0000","last_became_active":"2024-09-11T07:29:49.220032+0000","last_became_peered":"2024-09-11T07:29:49.220032+0000","last_unstale":"2024-09-11T07:29:59.192388+0000","last_undegraded":"2024-09-11T07:29:59.192388+0000","last_fullsized":"2024-09-11T07:29:59.192388+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:51:23.777665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.309638+0000","last_change":"2024-09-11T07:29:51.232066+0000","last_active":"2024-09-11T07:29:59.309638+0000","last_peered":"2024-09-11T07:29:59.309638+0000","last_clean":"2024-09-11T07:29:59.309638+0000","last_became_active":"2024-09-11T07:29:51.231779+0000","last_became_peered":"2024-09-11T07:29:51.231779+0000","last_unstale":"2024-09-11T07:29:59.309638+0000","last_undegraded":"2024-09-11T07:29:59.309638+0000","last_fullsized":"2024-09-11T07:29:59.309638+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:03.941638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185511+0000","last_change":"2024-09-11T07:29:53.244513+0000","last_active":"2024-09-11T07:29:59.185511+0000","last_peered":"2024-09-11T07:29:59.185511+0000","last_clean":"2024-09-11T07:29:59.185511+0000","last_became_active":"2024-09-11T07:29:53.244224+0000","last_became_peered":"2024-09-11T07:29:53.244224+0000","last_unstale":"2024-09-11T07:29:59.185511+0000","last_undegraded":"2024-09-11T07:29:59.185511+0000","last_fullsized":"2024-09-11T07:29:59.185511+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:47.625796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310995+0000","last_change":"2024-09-11T07:29:49.219858+0000","last_active":"2024-09-11T07:29:59.310995+0000","last_peered":"2024-09-11T07:29:59.310995+0000","last_clean":"2024-09-11T07:29:59.310995+0000","last_became_active":"2024-09-11T07:29:49.219556+0000","last_became_peered":"2024-09-11T07:29:49.219556+0000","last_unstale":"2024-09-11T07:29:59.310995+0000","last_undegraded":"2024-09-11T07:29:59.310995+0000","last_fullsized":"2024-09-11T07:29:59.310995+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:28:03.841375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185515+0000","last_change":"2024-09-11T07:29:51.228985+0000","last_active":"2024-09-11T07:29:59.185515+0000","last_peered":"2024-09-11T07:29:59.185515+0000","last_clean":"2024-09-11T07:29:59.185515+0000","last_became_active":"2024-09-11T07:29:51.228664+0000","last_became_peered":"2024-09-11T07:29:51.228664+0000","last_unstale":"2024-09-11T07:29:59.185515+0000","last_undegraded":"2024-09-11T07:29:59.185515+0000","last_fullsized":"2024-09-11T07:29:59.185515+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:23:40.462184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.190361+0000","last_change":"2024-09-11T07:29:53.240750+0000","last_active":"2024-09-11T07:29:59.190361+0000","last_peered":"2024-09-11T07:29:59.190361+0000","last_clean":"2024-09-11T07:29:59.190361+0000","last_became_active":"2024-09-11T07:29:53.238382+0000","last_became_peered":"2024-09-11T07:29:53.238382+0000","last_unstale":"2024-09-11T07:29:59.190361+0000","last_undegraded":"2024-09-11T07:29:59.190361+0000","last_fullsized":"2024-09-11T07:29:59.190361+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:45:53.512472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185692+0000","last_change":"2024-09-11T07:29:55.263334+0000","last_active":"2024-09-11T07:29:59.185692+0000","last_peered":"2024-09-11T07:29:59.185692+0000","last_clean":"2024-09-11T07:29:59.185692+0000","last_became_active":"2024-09-11T07:29:55.262985+0000","last_became_peered":"2024-09-11T07:29:55.262985+0000","last_unstale":"2024-09-11T07:29:59.185692+0000","last_undegraded":"2024-09-11T07:29:59.185692+0000","last_fullsized":"2024-09-11T07:29:59.185692+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:27:22.249161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427805+0000","last_change":"2024-09-11T07:29:49.219066+0000","last_active":"2024-09-11T07:29:59.427805+0000","last_peered":"2024-09-11T07:29:59.427805+0000","last_clean":"2024-09-11T07:29:59.427805+0000","last_became_active":"2024-09-11T07:29:49.218738+0000","last_became_peered":"2024-09-11T07:29:49.218738+0000","last_unstale":"2024-09-11T07:29:59.427805+0000","last_undegraded":"2024-09-11T07:29:59.427805+0000","last_fullsized":"2024-09-11T07:29:59.427805+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:01.493359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"60'5","reported_seq":26,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185806+0000","last_change":"2024-09-11T07:29:51.229295+0000","last_active":"2024-09-11T07:29:59.185806+0000","last_peered":"2024-09-11T07:29:59.185806+0000","last_clean":"2024-09-11T07:29:59.185806+0000","last_became_active":"2024-09-11T07:29:51.228974+0000","last_became_peered":"2024-09-11T07:29:51.228974+0000","last_unstale":"2024-09-11T07:29:59.185806+0000","last_undegraded":"2024-09-11T07:29:59.185806+0000","last_fullsized":"2024-09-11T07:29:59.185806+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:59.284205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427471+0000","last_change":"2024-09-11T07:29:53.244230+0000","last_active":"2024-09-11T07:29:59.427471+0000","last_peered":"2024-09-11T07:29:59.427471+0000","last_clean":"2024-09-11T07:29:59.427471+0000","last_became_active":"2024-09-11T07:29:53.243957+0000","last_became_peered":"2024-09-11T07:29:53.243957+0000","last_unstale":"2024-09-11T07:29:59.427471+0000","last_undegraded":"2024-09-11T07:29:59.427471+0000","last_fullsized":"2024-09-11T07:29:59.427471+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:47:06.409144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013189+0000","last_change":"2024-09-11T07:29:55.264649+0000","last_active":"2024-09-11T07:30:00.013189+0000","last_peered":"2024-09-11T07:30:00.013189+0000","last_clean":"2024-09-11T07:30:00.013189+0000","last_became_active":"2024-09-11T07:29:55.264376+0000","last_became_peered":"2024-09-11T07:29:55.264376+0000","last_unstale":"2024-09-11T07:30:00.013189+0000","last_undegraded":"2024-09-11T07:30:00.013189+0000","last_fullsized":"2024-09-11T07:30:00.013189+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:27:18.040554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.428037+0000","last_change":"2024-09-11T07:29:49.221507+0000","last_active":"2024-09-11T07:29:59.428037+0000","last_peered":"2024-09-11T07:29:59.428037+0000","last_clean":"2024-09-11T07:29:59.428037+0000","last_became_active":"2024-09-11T07:29:49.221264+0000","last_became_peered":"2024-09-11T07:29:49.221264+0000","last_unstale":"2024-09-11T07:29:59.428037+0000","last_undegraded":"2024-09-11T07:29:59.428037+0000","last_fullsized":"2024-09-11T07:29:59.428037+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:14:32.086856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185820+0000","last_change":"2024-09-11T07:29:51.229153+0000","last_active":"2024-09-11T07:29:59.185820+0000","last_peered":"2024-09-11T07:29:59.185820+0000","last_clean":"2024-09-11T07:29:59.185820+0000","last_became_active":"2024-09-11T07:29:51.228867+0000","last_became_peered":"2024-09-11T07:29:51.228867+0000","last_unstale":"2024-09-11T07:29:59.185820+0000","last_undegraded":"2024-09-11T07:29:59.185820+0000","last_fullsized":"2024-09-11T07:29:59.185820+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:44:15.343759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013288+0000","last_change":"2024-09-11T07:29:53.247558+0000","last_active":"2024-09-11T07:30:00.013288+0000","last_peered":"2024-09-11T07:30:00.013288+0000","last_clean":"2024-09-11T07:30:00.013288+0000","last_became_active":"2024-09-11T07:29:53.247227+0000","last_became_peered":"2024-09-11T07:29:53.247227+0000","last_unstale":"2024-09-11T07:30:00.013288+0000","last_undegraded":"2024-09-11T07:30:00.013288+0000","last_fullsized":"2024-09-11T07:30:00.013288+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:13.426173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191157+0000","last_change":"2024-09-11T07:29:55.267569+0000","last_active":"2024-09-11T07:29:59.191157+0000","last_peered":"2024-09-11T07:29:59.191157+0000","last_clean":"2024-09-11T07:29:59.191157+0000","last_became_active":"2024-09-11T07:29:55.267262+0000","last_became_peered":"2024-09-11T07:29:55.267262+0000","last_unstale":"2024-09-11T07:29:59.191157+0000","last_undegraded":"2024-09-11T07:29:59.191157+0000","last_fullsized":"2024-09-11T07:29:59.191157+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:27:01.693078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"60'17","reported_seq":44,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.186191+0000","last_change":"2024-09-11T07:29:51.229845+0000","last_active":"2024-09-11T07:29:59.186191+0000","last_peered":"2024-09-11T07:29:59.186191+0000","last_clean":"2024-09-11T07:29:59.186191+0000","last_became_active":"2024-09-11T07:29:51.229452+0000","last_became_peered":"2024-09-11T07:29:51.229452+0000","last_unstale":"2024-09-11T07:29:59.186191+0000","last_undegraded":"2024-09-11T07:29:59.186191+0000","last_fullsized":"2024-09-11T07:29:59.186191+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:12.853243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427887+0000","last_change":"2024-09-11T07:29:49.218197+0000","last_active":"2024-09-11T07:29:59.427887+0000","last_peered":"2024-09-11T07:29:59.427887+0000","last_clean":"2024-09-11T07:29:59.427887+0000","last_became_active":"2024-09-11T07:29:49.217928+0000","last_became_peered":"2024-09-11T07:29:49.217928+0000","last_unstale":"2024-09-11T07:29:59.427887+0000","last_undegraded":"2024-09-11T07:29:59.427887+0000","last_fullsized":"2024-09-11T07:29:59.427887+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:59:21.252858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191368+0000","last_change":"2024-09-11T07:29:53.248486+0000","last_active":"2024-09-11T07:29:59.191368+0000","last_peered":"2024-09-11T07:29:59.191368+0000","last_clean":"2024-09-11T07:29:59.191368+0000","last_became_active":"2024-09-11T07:29:53.248194+0000","last_became_peered":"2024-09-11T07:29:53.248194+0000","last_unstale":"2024-09-11T07:29:59.191368+0000","last_undegraded":"2024-09-11T07:29:59.191368+0000","last_fullsized":"2024-09-11T07:29:59.191368+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:26:20.557318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.601180+0000","last_change":"2024-09-11T07:29:55.266503+0000","last_active":"2024-09-11T07:29:59.601180+0000","last_peered":"2024-09-11T07:29:59.601180+0000","last_clean":"2024-09-11T07:29:59.601180+0000","last_became_active":"2024-09-11T07:29:55.266240+0000","last_became_peered":"2024-09-11T07:29:55.266240+0000","last_unstale":"2024-09-11T07:29:59.601180+0000","last_undegraded":"2024-09-11T07:29:59.601180+0000","last_fullsized":"2024-09-11T07:29:59.601180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:34:37.890636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185990+0000","last_change":"2024-09-11T07:29:51.228113+0000","last_active":"2024-09-11T07:29:59.185990+0000","last_peered":"2024-09-11T07:29:59.185990+0000","last_clean":"2024-09-11T07:29:59.185990+0000","last_became_active":"2024-09-11T07:29:51.227100+0000","last_became_peered":"2024-09-11T07:29:51.227100+0000","last_unstale":"2024-09-11T07:29:59.185990+0000","last_undegraded":"2024-09-11T07:29:59.185990+0000","last_fullsized":"2024-09-11T07:29:59.185990+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:38.948094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191086+0000","last_change":"2024-09-11T07:29:49.224400+0000","last_active":"2024-09-11T07:29:59.191086+0000","last_peered":"2024-09-11T07:29:59.191086+0000","last_clean":"2024-09-11T07:29:59.191086+0000","last_became_active":"2024-09-11T07:29:49.224025+0000","last_became_peered":"2024-09-11T07:29:49.224025+0000","last_unstale":"2024-09-11T07:29:59.191086+0000","last_undegraded":"2024-09-11T07:29:59.191086+0000","last_fullsized":"2024-09-11T07:29:59.191086+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:50:09.460643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"60'8","reported_seq":41,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457352+0000","last_change":"2024-09-11T07:29:53.240946+0000","last_active":"2024-09-11T07:31:02.457352+0000","last_peered":"2024-09-11T07:31:02.457352+0000","last_clean":"2024-09-11T07:31:02.457352+0000","last_became_active":"2024-09-11T07:29:53.240648+0000","last_became_peered":"2024-09-11T07:29:53.240648+0000","last_unstale":"2024-09-11T07:31:02.457352+0000","last_undegraded":"2024-09-11T07:31:02.457352+0000","last_fullsized":"2024-09-11T07:31:02.457352+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:04.275293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.186055+0000","last_change":"2024-09-11T07:29:55.267221+0000","last_active":"2024-09-11T07:29:59.186055+0000","last_peered":"2024-09-11T07:29:59.186055+0000","last_clean":"2024-09-11T07:29:59.186055+0000","last_became_active":"2024-09-11T07:29:55.266973+0000","last_became_peered":"2024-09-11T07:29:55.266973+0000","last_unstale":"2024-09-11T07:29:59.186055+0000","last_undegraded":"2024-09-11T07:29:59.186055+0000","last_fullsized":"2024-09-11T07:29:59.186055+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:00:11.369689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191847+0000","last_change":"2024-09-11T07:29:51.230533+0000","last_active":"2024-09-11T07:29:59.191847+0000","last_peered":"2024-09-11T07:29:59.191847+0000","last_clean":"2024-09-11T07:29:59.191847+0000","last_became_active":"2024-09-11T07:29:51.230207+0000","last_became_peered":"2024-09-11T07:29:51.230207+0000","last_unstale":"2024-09-11T07:29:59.191847+0000","last_undegraded":"2024-09-11T07:29:59.191847+0000","last_fullsized":"2024-09-11T07:29:59.191847+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:44.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.190536+0000","last_change":"2024-09-11T07:29:49.219305+0000","last_active":"2024-09-11T07:29:59.190536+0000","last_peered":"2024-09-11T07:29:59.190536+0000","last_clean":"2024-09-11T07:29:59.190536+0000","last_became_active":"2024-09-11T07:29:49.219056+0000","last_became_peered":"2024-09-11T07:29:49.219056+0000","last_unstale":"2024-09-11T07:29:59.190536+0000","last_undegraded":"2024-09-11T07:29:59.190536+0000","last_fullsized":"2024-09-11T07:29:59.190536+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:49:21.152009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457459+0000","last_change":"2024-09-11T07:29:53.246435+0000","last_active":"2024-09-11T07:31:02.457459+0000","last_peered":"2024-09-11T07:31:02.457459+0000","last_clean":"2024-09-11T07:31:02.457459+0000","last_became_active":"2024-09-11T07:29:53.246150+0000","last_became_peered":"2024-09-11T07:29:53.246150+0000","last_unstale":"2024-09-11T07:31:02.457459+0000","last_undegraded":"2024-09-11T07:31:02.457459+0000","last_fullsized":"2024-09-11T07:31:02.457459+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:39:34.124732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013513+0000","last_change":"2024-09-11T07:29:55.264617+0000","last_active":"2024-09-11T07:30:00.013513+0000","last_peered":"2024-09-11T07:30:00.013513+0000","last_clean":"2024-09-11T07:30:00.013513+0000","last_became_active":"2024-09-11T07:29:55.264321+0000","last_became_peered":"2024-09-11T07:29:55.264321+0000","last_unstale":"2024-09-11T07:30:00.013513+0000","last_undegraded":"2024-09-11T07:30:00.013513+0000","last_fullsized":"2024-09-11T07:30:00.013513+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:34:41.600998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.186191+0000","last_change":"2024-09-11T07:29:51.227712+0000","last_active":"2024-09-11T07:29:59.186191+0000","last_peered":"2024-09-11T07:29:59.186191+0000","last_clean":"2024-09-11T07:29:59.186191+0000","last_became_active":"2024-09-11T07:29:51.227366+0000","last_became_peered":"2024-09-11T07:29:51.227366+0000","last_unstale":"2024-09-11T07:29:59.186191+0000","last_undegraded":"2024-09-11T07:29:59.186191+0000","last_fullsized":"2024-09-11T07:29:59.186191+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:32.940047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427512+0000","last_change":"2024-09-11T07:29:49.220244+0000","last_active":"2024-09-11T07:29:59.427512+0000","last_peered":"2024-09-11T07:29:59.427512+0000","last_clean":"2024-09-11T07:29:59.427512+0000","last_became_active":"2024-09-11T07:29:49.219961+0000","last_became_peered":"2024-09-11T07:29:49.219961+0000","last_unstale":"2024-09-11T07:29:59.427512+0000","last_undegraded":"2024-09-11T07:29:59.427512+0000","last_fullsized":"2024-09-11T07:29:59.427512+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:50.888550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013648+0000","last_change":"2024-09-11T07:29:53.242774+0000","last_active":"2024-09-11T07:30:00.013648+0000","last_peered":"2024-09-11T07:30:00.013648+0000","last_clean":"2024-09-11T07:30:00.013648+0000","last_became_active":"2024-09-11T07:29:53.242484+0000","last_became_peered":"2024-09-11T07:29:53.242484+0000","last_unstale":"2024-09-11T07:30:00.013648+0000","last_undegraded":"2024-09-11T07:30:00.013648+0000","last_fullsized":"2024-09-11T07:30:00.013648+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:37:23.956005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310730+0000","last_change":"2024-09-11T07:29:55.328486+0000","last_active":"2024-09-11T07:29:59.310730+0000","last_peered":"2024-09-11T07:29:59.310730+0000","last_clean":"2024-09-11T07:29:59.310730+0000","last_became_active":"2024-09-11T07:29:55.328172+0000","last_became_peered":"2024-09-11T07:29:55.328172+0000","last_unstale":"2024-09-11T07:29:59.310730+0000","last_undegraded":"2024-09-11T07:29:59.310730+0000","last_fullsized":"2024-09-11T07:29:59.310730+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:03.124123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"60'14","reported_seq":37,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185642+0000","last_change":"2024-09-11T07:29:51.223754+0000","last_active":"2024-09-11T07:29:59.185642+0000","last_peered":"2024-09-11T07:29:59.185642+0000","last_clean":"2024-09-11T07:29:59.185642+0000","last_became_active":"2024-09-11T07:29:51.223386+0000","last_became_peered":"2024-09-11T07:29:51.223386+0000","last_unstale":"2024-09-11T07:29:59.185642+0000","last_undegraded":"2024-09-11T07:29:59.185642+0000","last_fullsized":"2024-09-11T07:29:59.185642+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:42:42.242532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.575134+0000","last_change":"2024-09-11T07:29:49.217087+0000","last_active":"2024-09-11T07:29:59.575134+0000","last_peered":"2024-09-11T07:29:59.575134+0000","last_clean":"2024-09-11T07:29:59.575134+0000","last_became_active":"2024-09-11T07:29:49.216808+0000","last_became_peered":"2024-09-11T07:29:49.216808+0000","last_unstale":"2024-09-11T07:29:59.575134+0000","last_undegraded":"2024-09-11T07:29:59.575134+0000","last_fullsized":"2024-09-11T07:29:59.575134+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:12.674171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457322+0000","last_change":"2024-09-11T07:29:53.252350+0000","last_active":"2024-09-11T07:31:02.457322+0000","last_peered":"2024-09-11T07:31:02.457322+0000","last_clean":"2024-09-11T07:31:02.457322+0000","last_became_active":"2024-09-11T07:29:53.251440+0000","last_became_peered":"2024-09-11T07:29:53.251440+0000","last_unstale":"2024-09-11T07:31:02.457322+0000","last_undegraded":"2024-09-11T07:31:02.457322+0000","last_fullsized":"2024-09-11T07:31:02.457322+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:24:14.998472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427652+0000","last_change":"2024-09-11T07:29:55.330660+0000","last_active":"2024-09-11T07:29:59.427652+0000","last_peered":"2024-09-11T07:29:59.427652+0000","last_clean":"2024-09-11T07:29:59.427652+0000","last_became_active":"2024-09-11T07:29:55.330307+0000","last_became_peered":"2024-09-11T07:29:55.330307+0000","last_unstale":"2024-09-11T07:29:59.427652+0000","last_undegraded":"2024-09-11T07:29:59.427652+0000","last_fullsized":"2024-09-11T07:29:59.427652+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:10:44.928896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"60'18","reported_seq":48,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.309549+0000","last_change":"2024-09-11T07:29:51.249652+0000","last_active":"2024-09-11T07:29:59.309549+0000","last_peered":"2024-09-11T07:29:59.309549+0000","last_clean":"2024-09-11T07:29:59.309549+0000","last_became_active":"2024-09-11T07:29:51.249369+0000","last_became_peered":"2024-09-11T07:29:51.249369+0000","last_unstale":"2024-09-11T07:29:59.309549+0000","last_undegraded":"2024-09-11T07:29:59.309549+0000","last_fullsized":"2024-09-11T07:29:59.309549+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:34:41.806338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191847+0000","last_change":"2024-09-11T07:29:49.212730+0000","last_active":"2024-09-11T07:29:59.191847+0000","last_peered":"2024-09-11T07:29:59.191847+0000","last_clean":"2024-09-11T07:29:59.191847+0000","last_became_active":"2024-09-11T07:29:49.212431+0000","last_became_peered":"2024-09-11T07:29:49.212431+0000","last_unstale":"2024-09-11T07:29:59.191847+0000","last_undegraded":"2024-09-11T07:29:59.191847+0000","last_fullsized":"2024-09-11T07:29:59.191847+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:49:59.391366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185565+0000","last_change":"2024-09-11T07:29:53.249204+0000","last_active":"2024-09-11T07:29:59.185565+0000","last_peered":"2024-09-11T07:29:59.185565+0000","last_clean":"2024-09-11T07:29:59.185565+0000","last_became_active":"2024-09-11T07:29:53.248928+0000","last_became_peered":"2024-09-11T07:29:53.248928+0000","last_unstale":"2024-09-11T07:29:59.185565+0000","last_undegraded":"2024-09-11T07:29:59.185565+0000","last_fullsized":"2024-09-11T07:29:59.185565+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:59:07.692517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185793+0000","last_change":"2024-09-11T07:29:55.258609+0000","last_active":"2024-09-11T07:29:59.185793+0000","last_peered":"2024-09-11T07:29:59.185793+0000","last_clean":"2024-09-11T07:29:59.185793+0000","last_became_active":"2024-09-11T07:29:55.258346+0000","last_became_peered":"2024-09-11T07:29:55.258346+0000","last_unstale":"2024-09-11T07:29:59.185793+0000","last_undegraded":"2024-09-11T07:29:59.185793+0000","last_fullsized":"2024-09-11T07:29:59.185793+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:48:30.919282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"60'19","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192456+0000","last_change":"2024-09-11T07:29:51.249559+0000","last_active":"2024-09-11T07:29:59.192456+0000","last_peered":"2024-09-11T07:29:59.192456+0000","last_clean":"2024-09-11T07:29:59.192456+0000","last_became_active":"2024-09-11T07:29:51.249303+0000","last_became_peered":"2024-09-11T07:29:51.249303+0000","last_unstale":"2024-09-11T07:29:59.192456+0000","last_undegraded":"2024-09-11T07:29:59.192456+0000","last_fullsized":"2024-09-11T07:29:59.192456+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:10:20.345316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427894+0000","last_change":"2024-09-11T07:29:49.222049+0000","last_active":"2024-09-11T07:29:59.427894+0000","last_peered":"2024-09-11T07:29:59.427894+0000","last_clean":"2024-09-11T07:29:59.427894+0000","last_became_active":"2024-09-11T07:29:49.221775+0000","last_became_peered":"2024-09-11T07:29:49.221775+0000","last_unstale":"2024-09-11T07:29:59.427894+0000","last_undegraded":"2024-09-11T07:29:59.427894+0000","last_fullsized":"2024-09-11T07:29:59.427894+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:34:45.527666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.574531+0000","last_change":"2024-09-11T07:29:53.248215+0000","last_active":"2024-09-11T07:29:59.574531+0000","last_peered":"2024-09-11T07:29:59.574531+0000","last_clean":"2024-09-11T07:29:59.574531+0000","last_became_active":"2024-09-11T07:29:53.247969+0000","last_became_peered":"2024-09-11T07:29:53.247969+0000","last_unstale":"2024-09-11T07:29:59.574531+0000","last_undegraded":"2024-09-11T07:29:59.574531+0000","last_fullsized":"2024-09-11T07:29:59.574531+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:42.535405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.190345+0000","last_change":"2024-09-11T07:29:55.328531+0000","last_active":"2024-09-11T07:29:59.190345+0000","last_peered":"2024-09-11T07:29:59.190345+0000","last_clean":"2024-09-11T07:29:59.190345+0000","last_became_active":"2024-09-11T07:29:55.328219+0000","last_became_peered":"2024-09-11T07:29:55.328219+0000","last_unstale":"2024-09-11T07:29:59.190345+0000","last_undegraded":"2024-09-11T07:29:59.190345+0000","last_fullsized":"2024-09-11T07:29:59.190345+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:14:43.652813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.190876+0000","last_change":"2024-09-11T07:29:51.230569+0000","last_active":"2024-09-11T07:29:59.190876+0000","last_peered":"2024-09-11T07:29:59.190876+0000","last_clean":"2024-09-11T07:29:59.190876+0000","last_became_active":"2024-09-11T07:29:51.230279+0000","last_became_peered":"2024-09-11T07:29:51.230279+0000","last_unstale":"2024-09-11T07:29:59.190876+0000","last_undegraded":"2024-09-11T07:29:59.190876+0000","last_fullsized":"2024-09-11T07:29:59.190876+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:04.053471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192064+0000","last_change":"2024-09-11T07:29:49.217145+0000","last_active":"2024-09-11T07:29:59.192064+0000","last_peered":"2024-09-11T07:29:59.192064+0000","last_clean":"2024-09-11T07:29:59.192064+0000","last_became_active":"2024-09-11T07:29:49.216859+0000","last_became_peered":"2024-09-11T07:29:49.216859+0000","last_unstale":"2024-09-11T07:29:59.192064+0000","last_undegraded":"2024-09-11T07:29:59.192064+0000","last_fullsized":"2024-09-11T07:29:59.192064+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:44.142846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457320+0000","last_change":"2024-09-11T07:29:53.242824+0000","last_active":"2024-09-11T07:31:02.457320+0000","last_peered":"2024-09-11T07:31:02.457320+0000","last_clean":"2024-09-11T07:31:02.457320+0000","last_became_active":"2024-09-11T07:29:53.242428+0000","last_became_peered":"2024-09-11T07:29:53.242428+0000","last_unstale":"2024-09-11T07:31:02.457320+0000","last_undegraded":"2024-09-11T07:31:02.457320+0000","last_fullsized":"2024-09-11T07:31:02.457320+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:11.714924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191040+0000","last_change":"2024-09-11T07:29:55.264537+0000","last_active":"2024-09-11T07:29:59.191040+0000","last_peered":"2024-09-11T07:29:59.191040+0000","last_clean":"2024-09-11T07:29:59.191040+0000","last_became_active":"2024-09-11T07:29:55.264268+0000","last_became_peered":"2024-09-11T07:29:55.264268+0000","last_unstale":"2024-09-11T07:29:59.191040+0000","last_undegraded":"2024-09-11T07:29:59.191040+0000","last_fullsized":"2024-09-11T07:29:59.191040+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:34:27.037273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"60'28","reported_seq":80,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457352+0000","last_change":"2024-09-11T07:29:51.229849+0000","last_active":"2024-09-11T07:31:02.457352+0000","last_peered":"2024-09-11T07:31:02.457352+0000","last_clean":"2024-09-11T07:31:02.457352+0000","last_became_active":"2024-09-11T07:29:51.229599+0000","last_became_peered":"2024-09-11T07:29:51.229599+0000","last_unstale":"2024-09-11T07:31:02.457352+0000","last_undegraded":"2024-09-11T07:31:02.457352+0000","last_fullsized":"2024-09-11T07:31:02.457352+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:02.005471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013430+0000","last_change":"2024-09-11T07:29:49.212467+0000","last_active":"2024-09-11T07:30:00.013430+0000","last_peered":"2024-09-11T07:30:00.013430+0000","last_clean":"2024-09-11T07:30:00.013430+0000","last_became_active":"2024-09-11T07:29:49.212218+0000","last_became_peered":"2024-09-11T07:29:49.212218+0000","last_unstale":"2024-09-11T07:30:00.013430+0000","last_undegraded":"2024-09-11T07:30:00.013430+0000","last_fullsized":"2024-09-11T07:30:00.013430+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:32:24.863053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191906+0000","last_change":"2024-09-11T07:29:53.252365+0000","last_active":"2024-09-11T07:29:59.191906+0000","last_peered":"2024-09-11T07:29:59.191906+0000","last_clean":"2024-09-11T07:29:59.191906+0000","last_became_active":"2024-09-11T07:29:53.251781+0000","last_became_peered":"2024-09-11T07:29:53.251781+0000","last_unstale":"2024-09-11T07:29:59.191906+0000","last_undegraded":"2024-09-11T07:29:59.191906+0000","last_fullsized":"2024-09-11T07:29:59.191906+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:41:46.205252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310888+0000","last_change":"2024-09-11T07:29:55.257021+0000","last_active":"2024-09-11T07:29:59.310888+0000","last_peered":"2024-09-11T07:29:59.310888+0000","last_clean":"2024-09-11T07:29:59.310888+0000","last_became_active":"2024-09-11T07:29:55.256640+0000","last_became_peered":"2024-09-11T07:29:55.256640+0000","last_unstale":"2024-09-11T07:29:59.310888+0000","last_undegraded":"2024-09-11T07:29:59.310888+0000","last_fullsized":"2024-09-11T07:29:59.310888+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:04:13.829670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"60'13","reported_seq":43,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191956+0000","last_change":"2024-09-11T07:29:51.229071+0000","last_active":"2024-09-11T07:29:59.191956+0000","last_peered":"2024-09-11T07:29:59.191956+0000","last_clean":"2024-09-11T07:29:59.191956+0000","last_became_active":"2024-09-11T07:29:51.228793+0000","last_became_peered":"2024-09-11T07:29:51.228793+0000","last_unstale":"2024-09-11T07:29:59.191956+0000","last_undegraded":"2024-09-11T07:29:59.191956+0000","last_fullsized":"2024-09-11T07:29:59.191956+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:21.473187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"61'5","reported_seq":102,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:10.255165+0000","last_change":"2024-09-11T07:29:53.233096+0000","last_active":"2024-09-11T07:31:10.255165+0000","last_peered":"2024-09-11T07:31:10.255165+0000","last_clean":"2024-09-11T07:31:10.255165+0000","last_became_active":"2024-09-11T07:29:49.212382+0000","last_became_peered":"2024-09-11T07:29:49.212382+0000","last_unstale":"2024-09-11T07:31:10.255165+0000","last_undegraded":"2024-09-11T07:31:10.255165+0000","last_fullsized":"2024-09-11T07:31:10.255165+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:10.898880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059387800000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.600650+0000","last_change":"2024-09-11T07:29:49.218737+0000","last_active":"2024-09-11T07:29:59.600650+0000","last_peered":"2024-09-11T07:29:59.600650+0000","last_clean":"2024-09-11T07:29:59.600650+0000","last_became_active":"2024-09-11T07:29:49.218493+0000","last_became_peered":"2024-09-11T07:29:49.218493+0000","last_unstale":"2024-09-11T07:29:59.600650+0000","last_undegraded":"2024-09-11T07:29:59.600650+0000","last_fullsized":"2024-09-11T07:29:59.600650+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:54:22.783964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.601196+0000","last_change":"2024-09-11T07:29:53.240224+0000","last_active":"2024-09-11T07:29:59.601196+0000","last_peered":"2024-09-11T07:29:59.601196+0000","last_clean":"2024-09-11T07:29:59.601196+0000","last_became_active":"2024-09-11T07:29:53.239937+0000","last_became_peered":"2024-09-11T07:29:53.239937+0000","last_unstale":"2024-09-11T07:29:59.601196+0000","last_undegraded":"2024-09-11T07:29:59.601196+0000","last_fullsized":"2024-09-11T07:29:59.601196+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:49:42.143133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427661+0000","last_change":"2024-09-11T07:29:55.328576+0000","last_active":"2024-09-11T07:29:59.427661+0000","last_peered":"2024-09-11T07:29:59.427661+0000","last_clean":"2024-09-11T07:29:59.427661+0000","last_became_active":"2024-09-11T07:29:55.328292+0000","last_became_peered":"2024-09-11T07:29:55.328292+0000","last_unstale":"2024-09-11T07:29:59.427661+0000","last_undegraded":"2024-09-11T07:29:59.427661+0000","last_fullsized":"2024-09-11T07:29:59.427661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:51.100385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"60'12","reported_seq":34,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192610+0000","last_change":"2024-09-11T07:29:51.233079+0000","last_active":"2024-09-11T07:29:59.192610+0000","last_peered":"2024-09-11T07:29:59.192610+0000","last_clean":"2024-09-11T07:29:59.192610+0000","last_became_active":"2024-09-11T07:29:51.232822+0000","last_became_peered":"2024-09-11T07:29:51.232822+0000","last_unstale":"2024-09-11T07:29:59.192610+0000","last_undegraded":"2024-09-11T07:29:59.192610+0000","last_fullsized":"2024-09-11T07:29:59.192610+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:26.858631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"55'1","reported_seq":24,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191337+0000","last_change":"2024-09-11T07:29:53.241535+0000","last_active":"2024-09-11T07:29:59.191337+0000","last_peered":"2024-09-11T07:29:59.191337+0000","last_clean":"2024-09-11T07:29:59.191337+0000","last_became_active":"2024-09-11T07:29:49.220131+0000","last_became_peered":"2024-09-11T07:29:49.220131+0000","last_unstale":"2024-09-11T07:29:59.191337+0000","last_undegraded":"2024-09-11T07:29:59.191337+0000","last_fullsized":"2024-09-11T07:29:59.191337+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:10.835176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049260989999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427504+0000","last_change":"2024-09-11T07:29:49.220533+0000","last_active":"2024-09-11T07:29:59.427504+0000","last_peered":"2024-09-11T07:29:59.427504+0000","last_clean":"2024-09-11T07:29:59.427504+0000","last_became_active":"2024-09-11T07:29:49.220271+0000","last_became_peered":"2024-09-11T07:29:49.220271+0000","last_unstale":"2024-09-11T07:29:59.427504+0000","last_undegraded":"2024-09-11T07:29:59.427504+0000","last_fullsized":"2024-09-11T07:29:59.427504+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:57.145971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013689+0000","last_change":"2024-09-11T07:29:53.247273+0000","last_active":"2024-09-11T07:30:00.013689+0000","last_peered":"2024-09-11T07:30:00.013689+0000","last_clean":"2024-09-11T07:30:00.013689+0000","last_became_active":"2024-09-11T07:29:53.246977+0000","last_became_peered":"2024-09-11T07:29:53.246977+0000","last_unstale":"2024-09-11T07:30:00.013689+0000","last_undegraded":"2024-09-11T07:30:00.013689+0000","last_fullsized":"2024-09-11T07:30:00.013689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:03:30.499901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191284+0000","last_change":"2024-09-11T07:29:55.264599+0000","last_active":"2024-09-11T07:29:59.191284+0000","last_peered":"2024-09-11T07:29:59.191284+0000","last_clean":"2024-09-11T07:29:59.191284+0000","last_became_active":"2024-09-11T07:29:55.264276+0000","last_became_peered":"2024-09-11T07:29:55.264276+0000","last_unstale":"2024-09-11T07:29:59.191284+0000","last_undegraded":"2024-09-11T07:29:59.191284+0000","last_fullsized":"2024-09-11T07:29:59.191284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:26.370166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"60'14","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.574159+0000","last_change":"2024-09-11T07:29:51.249515+0000","last_active":"2024-09-11T07:29:59.574159+0000","last_peered":"2024-09-11T07:29:59.574159+0000","last_clean":"2024-09-11T07:29:59.574159+0000","last_became_active":"2024-09-11T07:29:51.249236+0000","last_became_peered":"2024-09-11T07:29:51.249236+0000","last_unstale":"2024-09-11T07:29:59.574159+0000","last_undegraded":"2024-09-11T07:29:59.574159+0000","last_fullsized":"2024-09-11T07:29:59.574159+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:00:08.555358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"57'2","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310689+0000","last_change":"2024-09-11T07:29:53.244519+0000","last_active":"2024-09-11T07:29:59.310689+0000","last_peered":"2024-09-11T07:29:59.310689+0000","last_clean":"2024-09-11T07:29:59.310689+0000","last_became_active":"2024-09-11T07:29:49.216312+0000","last_became_peered":"2024-09-11T07:29:49.216312+0000","last_unstale":"2024-09-11T07:29:59.310689+0000","last_undegraded":"2024-09-11T07:29:59.310689+0000","last_fullsized":"2024-09-11T07:29:59.310689+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:09.996156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019793269999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"53'1","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013399+0000","last_change":"2024-09-11T07:29:49.219502+0000","last_active":"2024-09-11T07:30:00.013399+0000","last_peered":"2024-09-11T07:30:00.013399+0000","last_clean":"2024-09-11T07:30:00.013399+0000","last_became_active":"2024-09-11T07:29:49.219246+0000","last_became_peered":"2024-09-11T07:29:49.219246+0000","last_unstale":"2024-09-11T07:30:00.013399+0000","last_undegraded":"2024-09-11T07:30:00.013399+0000","last_fullsized":"2024-09-11T07:30:00.013399+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:09:50.493982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"62'79","reported_seq":52,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:30:28.046381+0000","last_change":"2024-09-11T07:29:38.923543+0000","last_active":"2024-09-11T07:30:28.046381+0000","last_peered":"2024-09-11T07:30:28.046381+0000","last_clean":"2024-09-11T07:30:28.046381+0000","last_became_active":"2024-09-11T07:29:38.917813+0000","last_became_peered":"2024-09-11T07:29:38.917813+0000","last_unstale":"2024-09-11T07:30:28.046381+0000","last_undegraded":"2024-09-11T07:30:28.046381+0000","last_fullsized":"2024-09-11T07:30:28.046381+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.984340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:45.984340+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.984340+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:58:43.411014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427949+0000","last_change":"2024-09-11T07:29:53.240076+0000","last_active":"2024-09-11T07:29:59.427949+0000","last_peered":"2024-09-11T07:29:59.427949+0000","last_clean":"2024-09-11T07:29:59.427949+0000","last_became_active":"2024-09-11T07:29:53.239779+0000","last_became_peered":"2024-09-11T07:29:53.239779+0000","last_unstale":"2024-09-11T07:29:59.427949+0000","last_undegraded":"2024-09-11T07:29:59.427949+0000","last_fullsized":"2024-09-11T07:29:59.427949+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:16.177970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"60'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013735+0000","last_change":"2024-09-11T07:29:55.259465+0000","last_active":"2024-09-11T07:30:00.013735+0000","last_peered":"2024-09-11T07:30:00.013735+0000","last_clean":"2024-09-11T07:30:00.013735+0000","last_became_active":"2024-09-11T07:29:55.259230+0000","last_became_peered":"2024-09-11T07:29:55.259230+0000","last_unstale":"2024-09-11T07:30:00.013735+0000","last_undegraded":"2024-09-11T07:30:00.013735+0000","last_fullsized":"2024-09-11T07:30:00.013735+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:50.889796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"60'19","reported_seq":47,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.574134+0000","last_change":"2024-09-11T07:29:51.249454+0000","last_active":"2024-09-11T07:29:59.574134+0000","last_peered":"2024-09-11T07:29:59.574134+0000","last_clean":"2024-09-11T07:29:59.574134+0000","last_became_active":"2024-09-11T07:29:51.249161+0000","last_became_peered":"2024-09-11T07:29:51.249161+0000","last_unstale":"2024-09-11T07:29:59.574134+0000","last_undegraded":"2024-09-11T07:29:59.574134+0000","last_fullsized":"2024-09-11T07:29:59.574134+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:04.426603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.601279+0000","last_change":"2024-09-11T07:29:49.213494+0000","last_active":"2024-09-11T07:29:59.601279+0000","last_peered":"2024-09-11T07:29:59.601279+0000","last_clean":"2024-09-11T07:29:59.601279+0000","last_became_active":"2024-09-11T07:29:49.213232+0000","last_became_peered":"2024-09-11T07:29:49.213232+0000","last_unstale":"2024-09-11T07:29:59.601279+0000","last_undegraded":"2024-09-11T07:29:59.601279+0000","last_fullsized":"2024-09-11T07:29:59.601279+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:34:40.475670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.600876+0000","last_change":"2024-09-11T07:29:53.247329+0000","last_active":"2024-09-11T07:29:59.600876+0000","last_peered":"2024-09-11T07:29:59.600876+0000","last_clean":"2024-09-11T07:29:59.600876+0000","last_became_active":"2024-09-11T07:29:53.245070+0000","last_became_peered":"2024-09-11T07:29:53.245070+0000","last_unstale":"2024-09-11T07:29:59.600876+0000","last_undegraded":"2024-09-11T07:29:59.600876+0000","last_fullsized":"2024-09-11T07:29:59.600876+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:44:18.389917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427582+0000","last_change":"2024-09-11T07:29:55.261521+0000","last_active":"2024-09-11T07:29:59.427582+0000","last_peered":"2024-09-11T07:29:59.427582+0000","last_clean":"2024-09-11T07:29:59.427582+0000","last_became_active":"2024-09-11T07:29:55.261246+0000","last_became_peered":"2024-09-11T07:29:55.261246+0000","last_unstale":"2024-09-11T07:29:59.427582+0000","last_undegraded":"2024-09-11T07:29:59.427582+0000","last_fullsized":"2024-09-11T07:29:59.427582+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:35.700697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192415+0000","last_change":"2024-09-11T07:29:51.234923+0000","last_active":"2024-09-11T07:29:59.192415+0000","last_peered":"2024-09-11T07:29:59.192415+0000","last_clean":"2024-09-11T07:29:59.192415+0000","last_became_active":"2024-09-11T07:29:51.234559+0000","last_became_peered":"2024-09-11T07:29:51.234559+0000","last_unstale":"2024-09-11T07:29:59.192415+0000","last_undegraded":"2024-09-11T07:29:59.192415+0000","last_fullsized":"2024-09-11T07:29:59.192415+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:53.995040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192193+0000","last_change":"2024-09-11T07:29:49.217242+0000","last_active":"2024-09-11T07:29:59.192193+0000","last_peered":"2024-09-11T07:29:59.192193+0000","last_clean":"2024-09-11T07:29:59.192193+0000","last_became_active":"2024-09-11T07:29:49.216626+0000","last_became_peered":"2024-09-11T07:29:49.216626+0000","last_unstale":"2024-09-11T07:29:59.192193+0000","last_undegraded":"2024-09-11T07:29:59.192193+0000","last_fullsized":"2024-09-11T07:29:59.192193+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:04.040175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.600799+0000","last_change":"2024-09-11T07:29:53.247105+0000","last_active":"2024-09-11T07:29:59.600799+0000","last_peered":"2024-09-11T07:29:59.600799+0000","last_clean":"2024-09-11T07:29:59.600799+0000","last_became_active":"2024-09-11T07:29:53.246297+0000","last_became_peered":"2024-09-11T07:29:53.246297+0000","last_unstale":"2024-09-11T07:29:59.600799+0000","last_undegraded":"2024-09-11T07:29:59.600799+0000","last_fullsized":"2024-09-11T07:29:59.600799+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:03:52.143718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191190+0000","last_change":"2024-09-11T07:29:55.267603+0000","last_active":"2024-09-11T07:29:59.191190+0000","last_peered":"2024-09-11T07:29:59.191190+0000","last_clean":"2024-09-11T07:29:59.191190+0000","last_became_active":"2024-09-11T07:29:55.267294+0000","last_became_peered":"2024-09-11T07:29:55.267294+0000","last_unstale":"2024-09-11T07:29:59.191190+0000","last_undegraded":"2024-09-11T07:29:59.191190+0000","last_fullsized":"2024-09-11T07:29:59.191190+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:01.503521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013396+0000","last_change":"2024-09-11T07:29:51.249934+0000","last_active":"2024-09-11T07:30:00.013396+0000","last_peered":"2024-09-11T07:30:00.013396+0000","last_clean":"2024-09-11T07:30:00.013396+0000","last_became_active":"2024-09-11T07:29:51.249601+0000","last_became_peered":"2024-09-11T07:29:51.249601+0000","last_unstale":"2024-09-11T07:30:00.013396+0000","last_undegraded":"2024-09-11T07:30:00.013396+0000","last_fullsized":"2024-09-11T07:30:00.013396+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:50:25.219576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.601286+0000","last_change":"2024-09-11T07:29:49.219024+0000","last_active":"2024-09-11T07:29:59.601286+0000","last_peered":"2024-09-11T07:29:59.601286+0000","last_clean":"2024-09-11T07:29:59.601286+0000","last_became_active":"2024-09-11T07:29:49.218770+0000","last_became_peered":"2024-09-11T07:29:49.218770+0000","last_unstale":"2024-09-11T07:29:59.601286+0000","last_undegraded":"2024-09-11T07:29:59.601286+0000","last_fullsized":"2024-09-11T07:29:59.601286+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:07:25.270993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457430+0000","last_change":"2024-09-11T07:29:53.246183+0000","last_active":"2024-09-11T07:31:02.457430+0000","last_peered":"2024-09-11T07:31:02.457430+0000","last_clean":"2024-09-11T07:31:02.457430+0000","last_became_active":"2024-09-11T07:29:53.244782+0000","last_became_peered":"2024-09-11T07:29:53.244782+0000","last_unstale":"2024-09-11T07:31:02.457430+0000","last_undegraded":"2024-09-11T07:31:02.457430+0000","last_fullsized":"2024-09-11T07:31:02.457430+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:15.347044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013196+0000","last_change":"2024-09-11T07:29:55.328350+0000","last_active":"2024-09-11T07:30:00.013196+0000","last_peered":"2024-09-11T07:30:00.013196+0000","last_clean":"2024-09-11T07:30:00.013196+0000","last_became_active":"2024-09-11T07:29:55.327990+0000","last_became_peered":"2024-09-11T07:29:55.327990+0000","last_unstale":"2024-09-11T07:30:00.013196+0000","last_undegraded":"2024-09-11T07:30:00.013196+0000","last_fullsized":"2024-09-11T07:30:00.013196+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:58.793358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"60'12","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185573+0000","last_change":"2024-09-11T07:29:51.229850+0000","last_active":"2024-09-11T07:29:59.185573+0000","last_peered":"2024-09-11T07:29:59.185573+0000","last_clean":"2024-09-11T07:29:59.185573+0000","last_became_active":"2024-09-11T07:29:51.229476+0000","last_became_peered":"2024-09-11T07:29:51.229476+0000","last_unstale":"2024-09-11T07:29:59.185573+0000","last_undegraded":"2024-09-11T07:29:59.185573+0000","last_fullsized":"2024-09-11T07:29:59.185573+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:47.473614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"53'2","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.186295+0000","last_change":"2024-09-11T07:29:49.219072+0000","last_active":"2024-09-11T07:29:59.186295+0000","last_peered":"2024-09-11T07:29:59.186295+0000","last_clean":"2024-09-11T07:29:59.186295+0000","last_became_active":"2024-09-11T07:29:49.218799+0000","last_became_peered":"2024-09-11T07:29:49.218799+0000","last_unstale":"2024-09-11T07:29:59.186295+0000","last_undegraded":"2024-09-11T07:29:59.186295+0000","last_fullsized":"2024-09-11T07:29:59.186295+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:19.018239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.600531+0000","last_change":"2024-09-11T07:29:53.249399+0000","last_active":"2024-09-11T07:29:59.600531+0000","last_peered":"2024-09-11T07:29:59.600531+0000","last_clean":"2024-09-11T07:29:59.600531+0000","last_became_active":"2024-09-11T07:29:53.249078+0000","last_became_peered":"2024-09-11T07:29:53.249078+0000","last_unstale":"2024-09-11T07:29:59.600531+0000","last_undegraded":"2024-09-11T07:29:59.600531+0000","last_fullsized":"2024-09-11T07:29:59.600531+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:39:03.017643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310756+0000","last_change":"2024-09-11T07:29:55.263490+0000","last_active":"2024-09-11T07:29:59.310756+0000","last_peered":"2024-09-11T07:29:59.310756+0000","last_clean":"2024-09-11T07:29:59.310756+0000","last_became_active":"2024-09-11T07:29:55.263152+0000","last_became_peered":"2024-09-11T07:29:55.263152+0000","last_unstale":"2024-09-11T07:29:59.310756+0000","last_undegraded":"2024-09-11T07:29:59.310756+0000","last_fullsized":"2024-09-11T07:29:59.310756+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:46.947000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192520+0000","last_change":"2024-09-11T07:29:51.250412+0000","last_active":"2024-09-11T07:29:59.192520+0000","last_peered":"2024-09-11T07:29:59.192520+0000","last_clean":"2024-09-11T07:29:59.192520+0000","last_became_active":"2024-09-11T07:29:51.250164+0000","last_became_peered":"2024-09-11T07:29:51.250164+0000","last_unstale":"2024-09-11T07:29:59.192520+0000","last_undegraded":"2024-09-11T07:29:59.192520+0000","last_fullsized":"2024-09-11T07:29:59.192520+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:18:16.491134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.600598+0000","last_change":"2024-09-11T07:29:49.214901+0000","last_active":"2024-09-11T07:29:59.600598+0000","last_peered":"2024-09-11T07:29:59.600598+0000","last_clean":"2024-09-11T07:29:59.600598+0000","last_became_active":"2024-09-11T07:29:49.214515+0000","last_became_peered":"2024-09-11T07:29:49.214515+0000","last_unstale":"2024-09-11T07:29:59.600598+0000","last_undegraded":"2024-09-11T07:29:59.600598+0000","last_fullsized":"2024-09-11T07:29:59.600598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:29:39.702003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.311099+0000","last_change":"2024-09-11T07:29:53.252723+0000","last_active":"2024-09-11T07:29:59.311099+0000","last_peered":"2024-09-11T07:29:59.311099+0000","last_clean":"2024-09-11T07:29:59.311099+0000","last_became_active":"2024-09-11T07:29:53.252437+0000","last_became_peered":"2024-09-11T07:29:53.252437+0000","last_unstale":"2024-09-11T07:29:59.311099+0000","last_undegraded":"2024-09-11T07:29:59.311099+0000","last_fullsized":"2024-09-11T07:29:59.311099+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:27.639699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.600956+0000","last_change":"2024-09-11T07:29:55.260575+0000","last_active":"2024-09-11T07:29:59.600956+0000","last_peered":"2024-09-11T07:29:59.600956+0000","last_clean":"2024-09-11T07:29:59.600956+0000","last_became_active":"2024-09-11T07:29:55.258857+0000","last_became_peered":"2024-09-11T07:29:55.258857+0000","last_unstale":"2024-09-11T07:29:59.600956+0000","last_undegraded":"2024-09-11T07:29:59.600956+0000","last_fullsized":"2024-09-11T07:29:59.600956+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:44:39.461312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.574494+0000","last_change":"2024-09-11T07:29:49.220644+0000","last_active":"2024-09-11T07:29:59.574494+0000","last_peered":"2024-09-11T07:29:59.574494+0000","last_clean":"2024-09-11T07:29:59.574494+0000","last_became_active":"2024-09-11T07:29:49.220396+0000","last_became_peered":"2024-09-11T07:29:49.220396+0000","last_unstale":"2024-09-11T07:29:59.574494+0000","last_undegraded":"2024-09-11T07:29:59.574494+0000","last_fullsized":"2024-09-11T07:29:59.574494+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:09:17.154821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"60'6","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.309705+0000","last_change":"2024-09-11T07:29:51.249491+0000","last_active":"2024-09-11T07:29:59.309705+0000","last_peered":"2024-09-11T07:29:59.309705+0000","last_clean":"2024-09-11T07:29:59.309705+0000","last_became_active":"2024-09-11T07:29:51.249250+0000","last_became_peered":"2024-09-11T07:29:51.249250+0000","last_unstale":"2024-09-11T07:29:59.309705+0000","last_undegraded":"2024-09-11T07:29:59.309705+0000","last_fullsized":"2024-09-11T07:29:59.309705+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:05:05.171133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013599+0000","last_change":"2024-09-11T07:29:53.247752+0000","last_active":"2024-09-11T07:30:00.013599+0000","last_peered":"2024-09-11T07:30:00.013599+0000","last_clean":"2024-09-11T07:30:00.013599+0000","last_became_active":"2024-09-11T07:29:53.247326+0000","last_became_peered":"2024-09-11T07:29:53.247326+0000","last_unstale":"2024-09-11T07:30:00.013599+0000","last_undegraded":"2024-09-11T07:30:00.013599+0000","last_fullsized":"2024-09-11T07:30:00.013599+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:58:37.862577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427580+0000","last_change":"2024-09-11T07:29:55.330684+0000","last_active":"2024-09-11T07:29:59.427580+0000","last_peered":"2024-09-11T07:29:59.427580+0000","last_clean":"2024-09-11T07:29:59.427580+0000","last_became_active":"2024-09-11T07:29:55.330269+0000","last_became_peered":"2024-09-11T07:29:55.330269+0000","last_unstale":"2024-09-11T07:29:59.427580+0000","last_undegraded":"2024-09-11T07:29:59.427580+0000","last_fullsized":"2024-09-11T07:29:59.427580+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:00.256204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.309933+0000","last_change":"2024-09-11T07:29:51.249466+0000","last_active":"2024-09-11T07:29:59.309933+0000","last_peered":"2024-09-11T07:29:59.309933+0000","last_clean":"2024-09-11T07:29:59.309933+0000","last_became_active":"2024-09-11T07:29:51.249175+0000","last_became_peered":"2024-09-11T07:29:51.249175+0000","last_unstale":"2024-09-11T07:29:59.309933+0000","last_undegraded":"2024-09-11T07:29:59.309933+0000","last_fullsized":"2024-09-11T07:29:59.309933+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:54.092983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013360+0000","last_change":"2024-09-11T07:29:49.222921+0000","last_active":"2024-09-11T07:30:00.013360+0000","last_peered":"2024-09-11T07:30:00.013360+0000","last_clean":"2024-09-11T07:30:00.013360+0000","last_became_active":"2024-09-11T07:29:49.222527+0000","last_became_peered":"2024-09-11T07:29:49.222527+0000","last_unstale":"2024-09-11T07:30:00.013360+0000","last_undegraded":"2024-09-11T07:30:00.013360+0000","last_fullsized":"2024-09-11T07:30:00.013360+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:10.176383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457456+0000","last_change":"2024-09-11T07:29:53.249033+0000","last_active":"2024-09-11T07:31:02.457456+0000","last_peered":"2024-09-11T07:31:02.457456+0000","last_clean":"2024-09-11T07:31:02.457456+0000","last_became_active":"2024-09-11T07:29:53.248776+0000","last_became_peered":"2024-09-11T07:29:53.248776+0000","last_unstale":"2024-09-11T07:31:02.457456+0000","last_undegraded":"2024-09-11T07:31:02.457456+0000","last_fullsized":"2024-09-11T07:31:02.457456+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:42.468271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192235+0000","last_change":"2024-09-11T07:29:55.268702+0000","last_active":"2024-09-11T07:29:59.192235+0000","last_peered":"2024-09-11T07:29:59.192235+0000","last_clean":"2024-09-11T07:29:59.192235+0000","last_became_active":"2024-09-11T07:29:55.268442+0000","last_became_peered":"2024-09-11T07:29:55.268442+0000","last_unstale":"2024-09-11T07:29:59.192235+0000","last_undegraded":"2024-09-11T07:29:59.192235+0000","last_fullsized":"2024-09-11T07:29:59.192235+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:57:37.788190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013624+0000","last_change":"2024-09-11T07:29:51.250427+0000","last_active":"2024-09-11T07:30:00.013624+0000","last_peered":"2024-09-11T07:30:00.013624+0000","last_clean":"2024-09-11T07:30:00.013624+0000","last_became_active":"2024-09-11T07:29:51.250080+0000","last_became_peered":"2024-09-11T07:29:51.250080+0000","last_unstale":"2024-09-11T07:30:00.013624+0000","last_undegraded":"2024-09-11T07:30:00.013624+0000","last_fullsized":"2024-09-11T07:30:00.013624+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:28:37.695830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192566+0000","last_change":"2024-09-11T07:29:49.221230+0000","last_active":"2024-09-11T07:29:59.192566+0000","last_peered":"2024-09-11T07:29:59.192566+0000","last_clean":"2024-09-11T07:29:59.192566+0000","last_became_active":"2024-09-11T07:29:49.220997+0000","last_became_peered":"2024-09-11T07:29:49.220997+0000","last_unstale":"2024-09-11T07:29:59.192566+0000","last_undegraded":"2024-09-11T07:29:59.192566+0000","last_fullsized":"2024-09-11T07:29:59.192566+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:51:30.322272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457578+0000","last_change":"2024-09-11T07:29:53.252769+0000","last_active":"2024-09-11T07:31:02.457578+0000","last_peered":"2024-09-11T07:31:02.457578+0000","last_clean":"2024-09-11T07:31:02.457578+0000","last_became_active":"2024-09-11T07:29:53.252492+0000","last_became_peered":"2024-09-11T07:29:53.252492+0000","last_unstale":"2024-09-11T07:31:02.457578+0000","last_undegraded":"2024-09-11T07:31:02.457578+0000","last_fullsized":"2024-09-11T07:31:02.457578+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:49:43.079885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.186086+0000","last_change":"2024-09-11T07:29:55.258735+0000","last_active":"2024-09-11T07:29:59.186086+0000","last_peered":"2024-09-11T07:29:59.186086+0000","last_clean":"2024-09-11T07:29:59.186086+0000","last_became_active":"2024-09-11T07:29:55.258364+0000","last_became_peered":"2024-09-11T07:29:55.258364+0000","last_unstale":"2024-09-11T07:29:59.186086+0000","last_undegraded":"2024-09-11T07:29:59.186086+0000","last_fullsized":"2024-09-11T07:29:59.186086+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:41.945015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.190538+0000","last_change":"2024-09-11T07:29:51.227078+0000","last_active":"2024-09-11T07:29:59.190538+0000","last_peered":"2024-09-11T07:29:59.190538+0000","last_clean":"2024-09-11T07:29:59.190538+0000","last_became_active":"2024-09-11T07:29:51.226767+0000","last_became_peered":"2024-09-11T07:29:51.226767+0000","last_unstale":"2024-09-11T07:29:59.190538+0000","last_undegraded":"2024-09-11T07:29:59.190538+0000","last_fullsized":"2024-09-11T07:29:59.190538+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:43:08.574720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.574661+0000","last_change":"2024-09-11T07:29:49.215959+0000","last_active":"2024-09-11T07:29:59.574661+0000","last_peered":"2024-09-11T07:29:59.574661+0000","last_clean":"2024-09-11T07:29:59.574661+0000","last_became_active":"2024-09-11T07:29:49.215548+0000","last_became_peered":"2024-09-11T07:29:49.215548+0000","last_unstale":"2024-09-11T07:29:59.574661+0000","last_undegraded":"2024-09-11T07:29:59.574661+0000","last_fullsized":"2024-09-11T07:29:59.574661+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:28:18.199146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310732+0000","last_change":"2024-09-11T07:29:53.253598+0000","last_active":"2024-09-11T07:29:59.310732+0000","last_peered":"2024-09-11T07:29:59.310732+0000","last_clean":"2024-09-11T07:29:59.310732+0000","last_became_active":"2024-09-11T07:29:53.253316+0000","last_became_peered":"2024-09-11T07:29:53.253316+0000","last_unstale":"2024-09-11T07:29:59.310732+0000","last_undegraded":"2024-09-11T07:29:59.310732+0000","last_fullsized":"2024-09-11T07:29:59.310732+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:15.550907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191010+0000","last_change":"2024-09-11T07:29:55.268457+0000","last_active":"2024-09-11T07:29:59.191010+0000","last_peered":"2024-09-11T07:29:59.191010+0000","last_clean":"2024-09-11T07:29:59.191010+0000","last_became_active":"2024-09-11T07:29:55.268043+0000","last_became_peered":"2024-09-11T07:29:55.268043+0000","last_unstale":"2024-09-11T07:29:59.191010+0000","last_undegraded":"2024-09-11T07:29:59.191010+0000","last_fullsized":"2024-09-11T07:29:59.191010+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:11:53.175252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185852+0000","last_change":"2024-09-11T07:29:51.227742+0000","last_active":"2024-09-11T07:29:59.185852+0000","last_peered":"2024-09-11T07:29:59.185852+0000","last_clean":"2024-09-11T07:29:59.185852+0000","last_became_active":"2024-09-11T07:29:51.227338+0000","last_became_peered":"2024-09-11T07:29:51.227338+0000","last_unstale":"2024-09-11T07:29:59.185852+0000","last_undegraded":"2024-09-11T07:29:59.185852+0000","last_fullsized":"2024-09-11T07:29:59.185852+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:58.190819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192121+0000","last_change":"2024-09-11T07:29:49.213948+0000","last_active":"2024-09-11T07:29:59.192121+0000","last_peered":"2024-09-11T07:29:59.192121+0000","last_clean":"2024-09-11T07:29:59.192121+0000","last_became_active":"2024-09-11T07:29:49.213688+0000","last_became_peered":"2024-09-11T07:29:49.213688+0000","last_unstale":"2024-09-11T07:29:59.192121+0000","last_undegraded":"2024-09-11T07:29:59.192121+0000","last_fullsized":"2024-09-11T07:29:59.192121+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:55:58.520871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.190407+0000","last_change":"2024-09-11T07:29:53.247092+0000","last_active":"2024-09-11T07:29:59.190407+0000","last_peered":"2024-09-11T07:29:59.190407+0000","last_clean":"2024-09-11T07:29:59.190407+0000","last_became_active":"2024-09-11T07:29:53.246734+0000","last_became_peered":"2024-09-11T07:29:53.246734+0000","last_unstale":"2024-09-11T07:29:59.190407+0000","last_undegraded":"2024-09-11T07:29:59.190407+0000","last_fullsized":"2024-09-11T07:29:59.190407+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:54.215825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310287+0000","last_change":"2024-09-11T07:29:55.265565+0000","last_active":"2024-09-11T07:29:59.310287+0000","last_peered":"2024-09-11T07:29:59.310287+0000","last_clean":"2024-09-11T07:29:59.310287+0000","last_became_active":"2024-09-11T07:29:55.265297+0000","last_became_peered":"2024-09-11T07:29:55.265297+0000","last_unstale":"2024-09-11T07:29:59.310287+0000","last_undegraded":"2024-09-11T07:29:59.310287+0000","last_fullsized":"2024-09-11T07:29:59.310287+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:08:24.061874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"60'4","reported_seq":22,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.309585+0000","last_change":"2024-09-11T07:29:51.228237+0000","last_active":"2024-09-11T07:29:59.309585+0000","last_peered":"2024-09-11T07:29:59.309585+0000","last_clean":"2024-09-11T07:29:59.309585+0000","last_became_active":"2024-09-11T07:29:51.227964+0000","last_became_peered":"2024-09-11T07:29:51.227964+0000","last_unstale":"2024-09-11T07:29:59.309585+0000","last_undegraded":"2024-09-11T07:29:59.309585+0000","last_fullsized":"2024-09-11T07:29:59.309585+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:09:44.080239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013694+0000","last_change":"2024-09-11T07:29:49.212211+0000","last_active":"2024-09-11T07:30:00.013694+0000","last_peered":"2024-09-11T07:30:00.013694+0000","last_clean":"2024-09-11T07:30:00.013694+0000","last_became_active":"2024-09-11T07:29:49.211946+0000","last_became_peered":"2024-09-11T07:29:49.211946+0000","last_unstale":"2024-09-11T07:30:00.013694+0000","last_undegraded":"2024-09-11T07:30:00.013694+0000","last_fullsized":"2024-09-11T07:30:00.013694+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:29:26.088357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.574391+0000","last_change":"2024-09-11T07:29:53.243979+0000","last_active":"2024-09-11T07:29:59.574391+0000","last_peered":"2024-09-11T07:29:59.574391+0000","last_clean":"2024-09-11T07:29:59.574391+0000","last_became_active":"2024-09-11T07:29:53.243683+0000","last_became_peered":"2024-09-11T07:29:53.243683+0000","last_unstale":"2024-09-11T07:29:59.574391+0000","last_undegraded":"2024-09-11T07:29:59.574391+0000","last_fullsized":"2024-09-11T07:29:59.574391+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:35:47.027425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427414+0000","last_change":"2024-09-11T07:29:55.258631+0000","last_active":"2024-09-11T07:29:59.427414+0000","last_peered":"2024-09-11T07:29:59.427414+0000","last_clean":"2024-09-11T07:29:59.427414+0000","last_became_active":"2024-09-11T07:29:55.258336+0000","last_became_peered":"2024-09-11T07:29:55.258336+0000","last_unstale":"2024-09-11T07:29:59.427414+0000","last_undegraded":"2024-09-11T07:29:59.427414+0000","last_fullsized":"2024-09-11T07:29:59.427414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:33.039223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310402+0000","last_change":"2024-09-11T07:29:51.250044+0000","last_active":"2024-09-11T07:29:59.310402+0000","last_peered":"2024-09-11T07:29:59.310402+0000","last_clean":"2024-09-11T07:29:59.310402+0000","last_became_active":"2024-09-11T07:29:51.249096+0000","last_became_peered":"2024-09-11T07:29:51.249096+0000","last_unstale":"2024-09-11T07:29:59.310402+0000","last_undegraded":"2024-09-11T07:29:59.310402+0000","last_fullsized":"2024-09-11T07:29:59.310402+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:57.444345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.575095+0000","last_change":"2024-09-11T07:29:49.216411+0000","last_active":"2024-09-11T07:29:59.575095+0000","last_peered":"2024-09-11T07:29:59.575095+0000","last_clean":"2024-09-11T07:29:59.575095+0000","last_became_active":"2024-09-11T07:29:49.216066+0000","last_became_peered":"2024-09-11T07:29:49.216066+0000","last_unstale":"2024-09-11T07:29:59.575095+0000","last_undegraded":"2024-09-11T07:29:59.575095+0000","last_fullsized":"2024-09-11T07:29:59.575095+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:24.592043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427312+0000","last_change":"2024-09-11T07:29:53.244412+0000","last_active":"2024-09-11T07:29:59.427312+0000","last_peered":"2024-09-11T07:29:59.427312+0000","last_clean":"2024-09-11T07:29:59.427312+0000","last_became_active":"2024-09-11T07:29:53.244161+0000","last_became_peered":"2024-09-11T07:29:53.244161+0000","last_unstale":"2024-09-11T07:29:59.427312+0000","last_undegraded":"2024-09-11T07:29:59.427312+0000","last_fullsized":"2024-09-11T07:29:59.427312+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:54:21.328805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310556+0000","last_change":"2024-09-11T07:29:55.329242+0000","last_active":"2024-09-11T07:29:59.310556+0000","last_peered":"2024-09-11T07:29:59.310556+0000","last_clean":"2024-09-11T07:29:59.310556+0000","last_became_active":"2024-09-11T07:29:55.328900+0000","last_became_peered":"2024-09-11T07:29:55.328900+0000","last_unstale":"2024-09-11T07:29:59.310556+0000","last_undegraded":"2024-09-11T07:29:59.310556+0000","last_fullsized":"2024-09-11T07:29:59.310556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:03.159527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"60'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427747+0000","last_change":"2024-09-11T07:29:55.261644+0000","last_active":"2024-09-11T07:29:59.427747+0000","last_peered":"2024-09-11T07:29:59.427747+0000","last_clean":"2024-09-11T07:29:59.427747+0000","last_became_active":"2024-09-11T07:29:55.261382+0000","last_became_peered":"2024-09-11T07:29:55.261382+0000","last_unstale":"2024-09-11T07:29:59.427747+0000","last_undegraded":"2024-09-11T07:29:59.427747+0000","last_fullsized":"2024-09-11T07:29:59.427747+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:40.086338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"53'1","reported_seq":27,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013146+0000","last_change":"2024-09-11T07:29:49.221452+0000","last_active":"2024-09-11T07:30:00.013146+0000","last_peered":"2024-09-11T07:30:00.013146+0000","last_clean":"2024-09-11T07:30:00.013146+0000","last_became_active":"2024-09-11T07:29:49.221186+0000","last_became_peered":"2024-09-11T07:29:49.221186+0000","last_unstale":"2024-09-11T07:30:00.013146+0000","last_undegraded":"2024-09-11T07:30:00.013146+0000","last_fullsized":"2024-09-11T07:30:00.013146+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:51:39.004180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192520+0000","last_change":"2024-09-11T07:29:51.251244+0000","last_active":"2024-09-11T07:29:59.192520+0000","last_peered":"2024-09-11T07:29:59.192520+0000","last_clean":"2024-09-11T07:29:59.192520+0000","last_became_active":"2024-09-11T07:29:51.251000+0000","last_became_peered":"2024-09-11T07:29:51.251000+0000","last_unstale":"2024-09-11T07:29:59.192520+0000","last_undegraded":"2024-09-11T07:29:59.192520+0000","last_fullsized":"2024-09-11T07:29:59.192520+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:32:08.729452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"60'8","reported_seq":41,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457457+0000","last_change":"2024-09-11T07:29:53.244528+0000","last_active":"2024-09-11T07:31:02.457457+0000","last_peered":"2024-09-11T07:31:02.457457+0000","last_clean":"2024-09-11T07:31:02.457457+0000","last_became_active":"2024-09-11T07:29:53.244235+0000","last_became_peered":"2024-09-11T07:29:53.244235+0000","last_unstale":"2024-09-11T07:31:02.457457+0000","last_undegraded":"2024-09-11T07:31:02.457457+0000","last_fullsized":"2024-09-11T07:31:02.457457+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:04:52.385829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191597+0000","last_change":"2024-09-11T07:29:55.263902+0000","last_active":"2024-09-11T07:29:59.191597+0000","last_peered":"2024-09-11T07:29:59.191597+0000","last_clean":"2024-09-11T07:29:59.191597+0000","last_became_active":"2024-09-11T07:29:55.263630+0000","last_became_peered":"2024-09-11T07:29:55.263630+0000","last_unstale":"2024-09-11T07:29:59.191597+0000","last_undegraded":"2024-09-11T07:29:59.191597+0000","last_fullsized":"2024-09-11T07:29:59.191597+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:26.902446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310537+0000","last_change":"2024-09-11T07:29:49.218872+0000","last_active":"2024-09-11T07:29:59.310537+0000","last_peered":"2024-09-11T07:29:59.310537+0000","last_clean":"2024-09-11T07:29:59.310537+0000","last_became_active":"2024-09-11T07:29:49.218553+0000","last_became_peered":"2024-09-11T07:29:49.218553+0000","last_unstale":"2024-09-11T07:29:59.310537+0000","last_undegraded":"2024-09-11T07:29:59.310537+0000","last_fullsized":"2024-09-11T07:29:59.310537+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:04:29.145363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.574275+0000","last_change":"2024-09-11T07:29:51.232028+0000","last_active":"2024-09-11T07:29:59.574275+0000","last_peered":"2024-09-11T07:29:59.574275+0000","last_clean":"2024-09-11T07:29:59.574275+0000","last_became_active":"2024-09-11T07:29:51.231721+0000","last_became_peered":"2024-09-11T07:29:51.231721+0000","last_unstale":"2024-09-11T07:29:59.574275+0000","last_undegraded":"2024-09-11T07:29:59.574275+0000","last_fullsized":"2024-09-11T07:29:59.574275+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:19.153130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192365+0000","last_change":"2024-09-11T07:29:53.251201+0000","last_active":"2024-09-11T07:29:59.192365+0000","last_peered":"2024-09-11T07:29:59.192365+0000","last_clean":"2024-09-11T07:29:59.192365+0000","last_became_active":"2024-09-11T07:29:53.250915+0000","last_became_peered":"2024-09-11T07:29:53.250915+0000","last_unstale":"2024-09-11T07:29:59.192365+0000","last_undegraded":"2024-09-11T07:29:59.192365+0000","last_fullsized":"2024-09-11T07:29:59.192365+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:56:21.448567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.186103+0000","last_change":"2024-09-11T07:29:55.328222+0000","last_active":"2024-09-11T07:29:59.186103+0000","last_peered":"2024-09-11T07:29:59.186103+0000","last_clean":"2024-09-11T07:29:59.186103+0000","last_became_active":"2024-09-11T07:29:55.327928+0000","last_became_peered":"2024-09-11T07:29:55.327928+0000","last_unstale":"2024-09-11T07:29:59.186103+0000","last_undegraded":"2024-09-11T07:29:59.186103+0000","last_fullsized":"2024-09-11T07:29:59.186103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:51:22.629378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.574543+0000","last_change":"2024-09-11T07:29:49.224002+0000","last_active":"2024-09-11T07:29:59.574543+0000","last_peered":"2024-09-11T07:29:59.574543+0000","last_clean":"2024-09-11T07:29:59.574543+0000","last_became_active":"2024-09-11T07:29:49.223718+0000","last_became_peered":"2024-09-11T07:29:49.223718+0000","last_unstale":"2024-09-11T07:29:59.574543+0000","last_undegraded":"2024-09-11T07:29:59.574543+0000","last_fullsized":"2024-09-11T07:29:59.574543+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:32.072683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.600246+0000","last_change":"2024-09-11T07:29:51.229685+0000","last_active":"2024-09-11T07:29:59.600246+0000","last_peered":"2024-09-11T07:29:59.600246+0000","last_clean":"2024-09-11T07:29:59.600246+0000","last_became_active":"2024-09-11T07:29:51.229440+0000","last_became_peered":"2024-09-11T07:29:51.229440+0000","last_unstale":"2024-09-11T07:29:59.600246+0000","last_undegraded":"2024-09-11T07:29:59.600246+0000","last_fullsized":"2024-09-11T07:29:59.600246+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:27:33.077762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191892+0000","last_change":"2024-09-11T07:29:53.248964+0000","last_active":"2024-09-11T07:29:59.191892+0000","last_peered":"2024-09-11T07:29:59.191892+0000","last_clean":"2024-09-11T07:29:59.191892+0000","last_became_active":"2024-09-11T07:29:53.246107+0000","last_became_peered":"2024-09-11T07:29:53.246107+0000","last_unstale":"2024-09-11T07:29:59.191892+0000","last_undegraded":"2024-09-11T07:29:59.191892+0000","last_fullsized":"2024-09-11T07:29:59.191892+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:17.721563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.311050+0000","last_change":"2024-09-11T07:29:55.328603+0000","last_active":"2024-09-11T07:29:59.311050+0000","last_peered":"2024-09-11T07:29:59.311050+0000","last_clean":"2024-09-11T07:29:59.311050+0000","last_became_active":"2024-09-11T07:29:55.328174+0000","last_became_peered":"2024-09-11T07:29:55.328174+0000","last_unstale":"2024-09-11T07:29:59.311050+0000","last_undegraded":"2024-09-11T07:29:59.311050+0000","last_fullsized":"2024-09-11T07:29:59.311050+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:20:21.186656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310883+0000","last_change":"2024-09-11T07:29:49.219357+0000","last_active":"2024-09-11T07:29:59.310883+0000","last_peered":"2024-09-11T07:29:59.310883+0000","last_clean":"2024-09-11T07:29:59.310883+0000","last_became_active":"2024-09-11T07:29:49.218848+0000","last_became_peered":"2024-09-11T07:29:49.218848+0000","last_unstale":"2024-09-11T07:29:59.310883+0000","last_undegraded":"2024-09-11T07:29:59.310883+0000","last_fullsized":"2024-09-11T07:29:59.310883+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:22:14.344430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"60'12","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.309644+0000","last_change":"2024-09-11T07:29:51.227143+0000","last_active":"2024-09-11T07:29:59.309644+0000","last_peered":"2024-09-11T07:29:59.309644+0000","last_clean":"2024-09-11T07:29:59.309644+0000","last_became_active":"2024-09-11T07:29:51.226709+0000","last_became_peered":"2024-09-11T07:29:51.226709+0000","last_unstale":"2024-09-11T07:29:59.309644+0000","last_undegraded":"2024-09-11T07:29:59.309644+0000","last_fullsized":"2024-09-11T07:29:59.309644+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:50:59.006747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185910+0000","last_change":"2024-09-11T07:29:53.248591+0000","last_active":"2024-09-11T07:29:59.185910+0000","last_peered":"2024-09-11T07:29:59.185910+0000","last_clean":"2024-09-11T07:29:59.185910+0000","last_became_active":"2024-09-11T07:29:53.248319+0000","last_became_peered":"2024-09-11T07:29:53.248319+0000","last_unstale":"2024-09-11T07:29:59.185910+0000","last_undegraded":"2024-09-11T07:29:59.185910+0000","last_fullsized":"2024-09-11T07:29:59.185910+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:27.733005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397525,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298544,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446608,"statfs":{"total":95995035648,"available":95689326592,"internally_reserved":0,"allocated":1220608,"data_stored":826179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.33700000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.36699999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.502}]},{"osd":4,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.52700000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.42399999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.439},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51500000000000001}]}]},{"osd":6,"up_from":43,"seq":184683593754,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298516,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446636,"statfs":{"total":95995035648,"available":95689355264,"internally_reserved":0,"allocated":1191936,"data_stored":824040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.436},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.44400000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.34599999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.46400000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.315}]},{"osd":5,"last update":"Wed Sep 11 07:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.36399999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.45200000000000001}]}]},{"osd":5,"up_from":37,"seq":158913789982,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297948,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447204,"statfs":{"total":95995035648,"available":95689936896,"internally_reserved":0,"allocated":606208,"data_stored":234542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.497,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.61899999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.38300000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.47399999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.624,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51300000000000001,"15min":0.51900000000000002},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.59599999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.622,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.36899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70299999999999996,"5min":0.751,"15min":0.751},"last":0.45100000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.497,"15min":0.495},"min":{"1min":0.40200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67500000000000004,"5min":0.748,"15min":0.748},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.48799999999999999,"15min":0.48899999999999999},"min":{"1min":0.374,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.52800000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.375,"15min":0.36899999999999999},"min":{"1min":0.26700000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.39300000000000002},"min":{"1min":0.28599999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63100000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.29099999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:30:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.27400000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.311},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.374}]}]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297968,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447184,"statfs":{"total":95995035648,"available":95689916416,"internally_reserved":0,"allocated":626688,"data_stored":234998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:30:29 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.51700000000000002,"15min":0.52700000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.63,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.73399999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.26400000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:30:29 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.504,"15min":0.50600000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70599999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70999999999999996,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.38200000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:30:29 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52300000000000002,"15min":0.52700000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69199999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.312,"5min":0.309,"15min":0.309},"max":{"1min":0.74399999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.33500000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:30:29 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.74399999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.68000000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.28699999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:30:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.33500000000000002},"min":{"1min":0.26900000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.60199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.34999999999999998,"15min":0.34000000000000002},"min":{"1min":0.31,"5min":0.255,"15min":0.255},"max":{"1min":0.55700000000000005,"5min":0.62,"15min":0.62},"last":0.51500000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:30:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.47599999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.502},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.433}]}]},{"osd":3,"up_from":26,"seq":111669149736,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297984,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447168,"statfs":{"total":95995035648,"available":95689900032,"internally_reserved":0,"allocated":643072,"data_stored":234790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:30:08 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.622,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.247}]},{"osd":1,"last update":"Wed Sep 11 07:30:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.40600000000000003,"15min":0.40699999999999997},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.51600000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.216},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.26600000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.52100000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.26500000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:30:08 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.438,"15min":0.442},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.54900000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40600000000000003,"15min":0.40699999999999997},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:30:33 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.42499999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:30:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35599999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:30:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.33400000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:30:40 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.373},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39300000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297948,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447204,"statfs":{"total":95995035648,"available":95689936896,"internally_reserved":0,"allocated":606208,"data_stored":234778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:30:53 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.37,"15min":0.38},"min":{"1min":0.216,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46999999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.314},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.39100000000000001,"15min":0.39900000000000002},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.53700000000000003,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.28399999999999997}]},{"osd":1,"last update":"Wed Sep 11 07:30:53 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.40000000000000002,"15min":0.40899999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60599999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.247},{"interface":"front","average":{"1min":0.378,"5min":0.40899999999999997,"15min":0.41899999999999998},"min":{"1min":0.26800000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.55700000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.41699999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:30:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.41799999999999998,"15min":0.42199999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.443,"15min":0.44900000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.59499999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.36799999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:30:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68300000000000005,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.40300000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:30:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53000000000000003,"15min":0.52800000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.37,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46000000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:30:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.54200000000000004}]},{"osd":7,"last update":"Wed Sep 11 07:30:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.55700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297996,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447156,"statfs":{"total":95995035648,"available":95689887744,"internally_reserved":0,"allocated":655360,"data_stored":235587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:30:29 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.33300000000000002,"15min":0.317},"min":{"1min":0.222,"5min":0.184,"15min":0.184},"max":{"1min":0.55600000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.20200000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.33800000000000002,"15min":0.32000000000000001},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.65100000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.26100000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:30:55 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.371,"15min":0.36499999999999999},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.52200000000000002,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.371,"15min":0.36499999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.53000000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.222}]},{"osd":3,"last update":"Wed Sep 11 07:30:11 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.53400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.248},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.438,"15min":0.441},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64000000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.32400000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:30:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.746,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.72199999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.65400000000000003}]},{"osd":5,"last update":"Wed Sep 11 07:30:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.53300000000000003,"15min":0.54000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72899999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.35199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84099999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.49099999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:30:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.628},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57199999999999995}]},{"osd":7,"last update":"Wed Sep 11 07:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.55000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298536,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446616,"statfs":{"total":95995035648,"available":95689334784,"internally_reserved":0,"allocated":1212416,"data_stored":824640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34100000000000003,"15min":0.33000000000000002},"min":{"1min":0.27600000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.46600000000000003,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.375},{"interface":"front","average":{"1min":0.379,"5min":0.35799999999999998,"15min":0.34399999999999997},"min":{"1min":0.26500000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.42999999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.38600000000000001,"15min":0.38700000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.46400000000000002,"5min":0.62,"15min":0.62},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.38200000000000001,"15min":0.378},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48599999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.50800000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:30:13 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.32500000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41099999999999998,"15min":0.41299999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.48599999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69699999999999995,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55600000000000005,"15min":0.56100000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.65500000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.53400000000000003}]},{"osd":5,"last update":"Wed Sep 11 07:30:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.65900000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.747,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.56299999999999994}]},{"osd":6,"last update":"Wed Sep 11 07:30:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58499999999999996}]},{"osd":7,"last update":"Wed Sep 11 07:30:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.626},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.59899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:31:13.874 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:31:14.115 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:14.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:14 smithi164 ceph-mon[33167]: from='client.24572 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:31:14.236 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:14 smithi143 ceph-mon[29604]: from='client.24572 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:31:14.236 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:13 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[73059]: ts=2024-09-11T07:31:13.871Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002859228s 2024-09-11T07:31:14.236 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:14 smithi143 ceph-mon[38451]: from='client.24572 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:31:14.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:31:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:31:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:31:15.260 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:15 smithi143 ceph-mon[29604]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:15.260 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:15 smithi143 ceph-mon[38451]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:15.361 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-mon[33167]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:15.361 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 systemd[1]: Starting Ceph grafana.a for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:31:15.680 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 podman[66771]: 2024-09-11 07:31:15.359463826 +0000 UTC m=+0.091941229 container create c3571985e0e2784fd0d9f321b39df2237126af1082d6305fe68a8c1653efb194 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a, release=239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.component=ubi8-container, name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.tags=base rhel8, maintainer=Paul Cuzner , architecture=x86_64, vendor=Red Hat, Inc., io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, io.buildah.version=1.32.0, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, vcs-type=git, io.openshift.expose-services=, description=Ceph Grafana Container, distribution-scope=public) 2024-09-11T07:31:15.681 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 podman[66771]: 2024-09-11 07:31:15.283238466 +0000 UTC m=+0.015715881 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-11T07:31:15.681 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 podman[66771]: 2024-09-11 07:31:15.603285627 +0000 UTC m=+0.335763034 container init c3571985e0e2784fd0d9f321b39df2237126af1082d6305fe68a8c1653efb194 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a, description=Ceph Grafana Container, build-date=2022-04-29T11:48:11.055897, com.redhat.component=ubi8-container, io.openshift.expose-services=, io.k8s.display-name=Red Hat Universal Base Image 8, name=ubi8, distribution-scope=public, maintainer=Paul Cuzner , architecture=x86_64, vendor=Red Hat, Inc., release=239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.buildah.version=1.32.0, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.tags=base rhel8, vcs-type=git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8.5) 2024-09-11T07:31:15.681 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 podman[66771]: 2024-09-11 07:31:15.607837305 +0000 UTC m=+0.340314709 container start c3571985e0e2784fd0d9f321b39df2237126af1082d6305fe68a8c1653efb194 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a, io.openshift.tags=base rhel8, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi8-container, io.buildah.version=1.32.0, vcs-type=git, io.openshift.expose-services=, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8.5, vendor=Red Hat, Inc., io.k8s.display-name=Red Hat Universal Base Image 8, distribution-scope=public, maintainer=Paul Cuzner , architecture=x86_64, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, release=239.1651231664, build-date=2022-04-29T11:48:11.055897, description=Ceph Grafana Container, summary=Grafana Container configured for Ceph mgr/dashboard integration) 2024-09-11T07:31:15.681 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 bash[66771]: c3571985e0e2784fd0d9f321b39df2237126af1082d6305fe68a8c1653efb194 2024-09-11T07:31:15.886 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:31:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-09-11T07:31:15.930 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 systemd[1]: Started Ceph grafana.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:31:15.930 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=settings t=2024-09-11T07:31:15.766292206Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-11T07:31:15.930 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=settings t=2024-09-11T07:31:15.76648304Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-11T07:31:15.930 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=settings t=2024-09-11T07:31:15.766508713Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-11T07:31:15.930 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=settings t=2024-09-11T07:31:15.766516791Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-11T07:31:15.930 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=settings t=2024-09-11T07:31:15.766523511Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-11T07:31:15.930 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=settings t=2024-09-11T07:31:15.766530574Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-11T07:31:15.931 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=settings t=2024-09-11T07:31:15.766537141Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-11T07:31:15.931 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=settings t=2024-09-11T07:31:15.76654309Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-11T07:31:15.931 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=settings t=2024-09-11T07:31:15.766549327Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-11T07:31:15.931 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=settings t=2024-09-11T07:31:15.766555169Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-11T07:31:15.931 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=settings t=2024-09-11T07:31:15.766560633Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-11T07:31:15.931 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=settings t=2024-09-11T07:31:15.766566217Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-11T07:31:15.931 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=settings t=2024-09-11T07:31:15.766571852Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-11T07:31:15.931 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=settings t=2024-09-11T07:31:15.766577577Z level=info msg="App mode production" 2024-09-11T07:31:15.931 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=sqlstore t=2024-09-11T07:31:15.766624379Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-11T07:31:15.931 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=sqlstore t=2024-09-11T07:31:15.766640958Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-11T07:31:15.931 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.767486673Z level=info msg="Starting DB migrations" 2024-09-11T07:31:15.932 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.767558033Z level=info msg="Executing migration" id="create migration_log table" 2024-09-11T07:31:15.933 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.768100086Z level=info msg="Executing migration" id="create user table" 2024-09-11T07:31:15.933 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.768562943Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-11T07:31:15.933 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.76903206Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-11T07:31:15.933 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.771807104Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-11T07:31:15.933 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.774535341Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-11T07:31:15.933 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.777398425Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-11T07:31:15.933 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.779997432Z level=info msg="Executing migration" id="create user table v2" 2024-09-11T07:31:15.933 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.782661742Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-11T07:31:15.933 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.785337266Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-11T07:31:15.933 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.788130654Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-11T07:31:15.933 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.790742566Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-11T07:31:15.933 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.793499167Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-11T07:31:15.934 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.794092311Z level=info msg="Executing migration" id="Update user table charset" 2024-09-11T07:31:15.934 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.796745945Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-11T07:31:15.934 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.799318549Z level=info msg="Executing migration" id="Add missing user data" 2024-09-11T07:31:15.934 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.802052042Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-11T07:31:15.934 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.802576939Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-11T07:31:15.934 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.805331774Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-11T07:31:15.934 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.808244733Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-11T07:31:15.934 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.810818324Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-11T07:31:15.934 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.811260513Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-11T07:31:15.934 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.81400925Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-11T07:31:15.934 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.816485424Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-11T07:31:15.934 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.816932406Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-11T07:31:15.934 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.817742945Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-11T07:31:15.935 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.818632581Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-11T07:31:15.935 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.81934236Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-11T07:31:15.935 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.820269506Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-11T07:31:15.935 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.820803676Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-11T07:31:15.935 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.82122101Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-11T07:31:15.935 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.822045842Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-11T07:31:15.935 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.823000052Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-11T07:31:15.935 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.823746287Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-11T07:31:15.935 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.824654987Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-11T07:31:15.935 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.825397021Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-11T07:31:15.935 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.826250225Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-11T07:31:15.935 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.827085256Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-11T07:31:15.935 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.827840019Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-11T07:31:15.936 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.828211587Z level=info msg="Executing migration" id="create star table" 2024-09-11T07:31:15.936 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.828995963Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-11T07:31:15.936 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.829481858Z level=info msg="Executing migration" id="create org table v1" 2024-09-11T07:31:15.936 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.829897312Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-11T07:31:15.936 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.830752065Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-11T07:31:15.936 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.831649256Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-11T07:31:15.936 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.832443692Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-11T07:31:15.936 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.833367253Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-11T07:31:15.936 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.833863123Z level=info msg="Executing migration" id="Update org table charset" 2024-09-11T07:31:15.936 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.834672304Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-11T07:31:15.936 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.835486808Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-11T07:31:15.936 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.836315178Z level=info msg="Executing migration" id="create dashboard table" 2024-09-11T07:31:15.937 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.83671469Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-11T07:31:15.937 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.837143126Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-11T07:31:15.937 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.838014349Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-11T07:31:15.937 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.838881291Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-11T07:31:15.937 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.83982933Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-11T07:31:15.937 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.840416594Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-11T07:31:15.937 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.841313304Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-11T07:31:15.937 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.842315093Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-11T07:31:15.937 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.845407599Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-11T07:31:15.937 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.845925182Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-11T07:31:15.937 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.848605501Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-11T07:31:15.937 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.851225281Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-11T07:31:15.938 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.85426463Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-11T07:31:15.938 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.85481118Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-11T07:31:15.938 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.857596647Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-11T07:31:15.938 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.860306787Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-11T07:31:15.938 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.863015699Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-11T07:31:15.938 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.863560775Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-11T07:31:15.938 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.866445525Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-11T07:31:15.938 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.869315092Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-11T07:31:15.938 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.871994205Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-11T07:31:15.938 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.872303659Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-11T07:31:15.938 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.87506259Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-11T07:31:15.938 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.877856014Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-11T07:31:15.938 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.880700504Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-11T07:31:15.938 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.88350654Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-11T07:31:15.939 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.886097497Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-11T07:31:15.939 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.888598146Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-11T07:31:15.939 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.891439864Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-11T07:31:15.939 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.891804696Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-11T07:31:15.939 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.894802877Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-11T07:31:15.939 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.897138907Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-11T07:31:15.939 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.899828185Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-11T07:31:15.939 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.900263905Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-11T07:31:15.939 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.903024549Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-11T07:31:15.939 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.905791769Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-11T07:31:15.939 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.90824616Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-11T07:31:15.939 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.908644519Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-11T07:31:15.939 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.911399335Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-11T07:31:15.939 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.914069389Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-11T07:31:15.939 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.916804231Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-11T07:31:15.940 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.917281145Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-11T07:31:15.940 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.920100741Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-11T07:31:15.940 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.922577377Z level=info msg="Executing migration" id="create data_source table" 2024-09-11T07:31:15.940 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.925242466Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-11T07:31:15.940 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.92571706Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-11T07:31:15.940 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.92618678Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-11T07:31:15.940 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.928770315Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-11T07:31:15.940 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.929338962Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-11T07:31:15.940 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.931632136Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-11T07:31:16.180 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.932216806Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-11T07:31:16.181 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.934821651Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-11T07:31:16.181 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.935387036Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-11T07:31:16.181 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.935844285Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-11T07:31:16.181 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.936590742Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-11T07:31:16.181 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.937363825Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-11T07:31:16.181 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.93776188Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-11T07:31:16.181 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.938607836Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-11T07:31:16.181 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.939425592Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-11T07:31:16.181 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.940245759Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-11T07:31:16.181 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.940574213Z level=info msg="Executing migration" id="Add uid column" 2024-09-11T07:31:16.181 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.941385509Z level=info msg="Executing migration" id="Update uid value" 2024-09-11T07:31:16.181 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.942193989Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-11T07:31:16.182 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.943016521Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-11T07:31:16.182 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.943462808Z level=info msg="Executing migration" id="create api_key table" 2024-09-11T07:31:16.182 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.944305816Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-11T07:31:16.182 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.945161588Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-11T07:31:16.182 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.945913205Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-11T07:31:16.182 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.946648267Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-11T07:31:16.183 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.947578987Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-11T07:31:16.183 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.948310261Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-11T07:31:16.183 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.949162379Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-11T07:31:16.183 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.950388755Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-11T07:31:16.183 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.950791162Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-11T07:31:16.183 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.951195865Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-11T07:31:16.183 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.951591798Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-11T07:31:16.183 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.952001283Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-11T07:31:16.183 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.95232453Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-11T07:31:16.184 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.952699307Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-11T07:31:16.184 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.952997812Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-11T07:31:16.184 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.953506154Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-11T07:31:16.184 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.954113362Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-11T07:31:16.184 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.954420536Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-11T07:31:16.184 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.954977612Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-11T07:31:16.184 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.955541487Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-11T07:31:16.184 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.955930257Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-11T07:31:16.184 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.95628591Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-11T07:31:16.184 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.956640581Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-11T07:31:16.184 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.957050141Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-11T07:31:16.184 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.957451603Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-11T07:31:16.184 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.957864297Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-11T07:31:16.184 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.95815145Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-11T07:31:16.185 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.958406145Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-11T07:31:16.185 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.958978731Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-11T07:31:16.185 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.959542962Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-11T07:31:16.185 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.959855499Z level=info msg="Executing migration" id="create quota table v1" 2024-09-11T07:31:16.185 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.960261837Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-11T07:31:16.185 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.960691063Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-11T07:31:16.185 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.961000366Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-11T07:31:16.185 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.961384842Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-11T07:31:16.185 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.961789699Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-11T07:31:16.185 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.962368248Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-11T07:31:16.185 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.962633562Z level=info msg="Executing migration" id="create session table" 2024-09-11T07:31:16.185 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.963070364Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-11T07:31:16.185 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.963358735Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-11T07:31:16.185 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.9636234Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-11T07:31:16.186 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.964009827Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-11T07:31:16.186 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.964402334Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-11T07:31:16.186 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.964656886Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-11T07:31:16.186 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.964946596Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-11T07:31:16.186 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.965212489Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-11T07:31:16.186 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.965480413Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-11T07:31:16.186 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.965878239Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-11T07:31:16.186 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.966160383Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-11T07:31:16.186 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.966724743Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-11T07:31:16.186 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.967031157Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-11T07:31:16.186 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.967583113Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-11T07:31:16.186 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.968276116Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-11T07:31:16.186 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.968584205Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-11T07:31:16.186 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.969077164Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-11T07:31:16.187 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.969557972Z level=info msg="Executing migration" id="create alert table v1" 2024-09-11T07:31:16.187 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.970372682Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-11T07:31:16.187 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.971361331Z level=info msg="Executing migration" id="add index alert state" 2024-09-11T07:31:16.187 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.971921558Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-11T07:31:16.187 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.972326Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-11T07:31:16.187 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.972692455Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-11T07:31:16.187 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.97314733Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-11T07:31:16.187 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.973505909Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-11T07:31:16.187 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.974927847Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-11T07:31:16.187 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.975327245Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-11T07:31:16.187 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.975953415Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-11T07:31:16.187 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.976264999Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-11T07:31:16.187 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.976587614Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-11T07:31:16.187 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.976976606Z level=info msg="Executing migration" id="Add column is_default" 2024-09-11T07:31:16.188 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.97760811Z level=info msg="Executing migration" id="Add column frequency" 2024-09-11T07:31:16.188 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.978264792Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-11T07:31:16.188 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.978894984Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-11T07:31:16.188 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.979528602Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-11T07:31:16.188 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.979940587Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-11T07:31:16.188 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.980247749Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-11T07:31:16.188 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.980509916Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-11T07:31:16.188 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.980899232Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-11T07:31:16.188 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.981329634Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-11T07:31:16.188 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.981673237Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-11T07:31:16.188 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.982153201Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-11T07:31:16.188 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.982564916Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-11T07:31:16.188 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.983260943Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-11T07:31:16.189 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.983932994Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-11T07:31:16.189 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.9848073Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-11T07:31:16.189 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.985891304Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-11T07:31:16.189 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.986611218Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-11T07:31:16.189 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.987418345Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-11T07:31:16.189 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.988344953Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-11T07:31:16.189 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.98917967Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-11T07:31:16.189 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.992110576Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-11T07:31:16.189 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.994965Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-11T07:31:16.189 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:15.997808449Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-11T07:31:16.189 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.000347942Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-11T07:31:16.189 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.003131738Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-11T07:31:16.189 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.006031127Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-11T07:31:16.189 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.008868132Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-11T07:31:16.190 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.011501609Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-11T07:31:16.190 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.014323607Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-11T07:31:16.190 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.017238164Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-11T07:31:16.190 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.019867996Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-11T07:31:16.190 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.022895142Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-11T07:31:16.190 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.025658872Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-11T07:31:16.190 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.028515433Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-11T07:31:16.190 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.031475567Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-11T07:31:16.190 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.034397159Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-11T07:31:16.190 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.03702819Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-11T07:31:16.190 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.03988489Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-11T07:31:16.190 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.042514739Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-11T07:31:16.190 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.045266908Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-11T07:31:16.190 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.047877104Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-11T07:31:16.191 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.050626728Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-11T07:31:16.191 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.053320239Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-11T07:31:16.191 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.056045365Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-11T07:31:16.191 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.058799863Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-11T07:31:16.191 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.0611694Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-11T07:31:16.191 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.061876547Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-11T07:31:16.191 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.064446653Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-11T07:31:16.191 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.06731144Z level=info msg="Executing migration" id="Move region to single row" 2024-09-11T07:31:16.191 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.068061421Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-11T07:31:16.192 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.068924653Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-11T07:31:16.192 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.069648833Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-11T07:31:16.192 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.070461006Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-11T07:31:16.192 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.071358214Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-11T07:31:16.192 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.072136376Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-11T07:31:16.193 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.073075048Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-11T07:31:16.193 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.073761814Z level=info msg="Executing migration" id="create test_data table" 2024-09-11T07:31:16.193 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.07453423Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-11T07:31:16.193 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.075497668Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-11T07:31:16.193 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.076245366Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-11T07:31:16.193 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.077026663Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-11T07:31:16.193 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.077837283Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-11T07:31:16.193 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.078860017Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-11T07:31:16.193 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.07957586Z level=info msg="Executing migration" id="create team table" 2024-09-11T07:31:16.193 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.08039255Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-11T07:31:16.193 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.081223029Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-11T07:31:16.193 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.082180761Z level=info msg="Executing migration" id="create team member table" 2024-09-11T07:31:16.193 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.082817748Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-11T07:31:16.193 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.083631371Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-11T07:31:16.194 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.08450987Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-11T07:31:16.194 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.085341859Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-11T07:31:16.194 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.086166358Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-11T07:31:16.194 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.086974023Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-11T07:31:16.194 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.087813311Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-11T07:31:16.194 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.088645661Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-11T07:31:16.194 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.089655519Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-11T07:31:16.194 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.090406563Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-11T07:31:16.194 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.091419186Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-11T07:31:16.194 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.092307686Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-11T07:31:16.194 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.093045047Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-11T07:31:16.194 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.093867098Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-11T07:31:16.194 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.094275581Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-11T07:31:16.194 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.095134834Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-11T07:31:16.195 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.095465648Z level=info msg="Executing migration" id="create tag table" 2024-09-11T07:31:16.195 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.095998349Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-11T07:31:16.195 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.098827678Z level=info msg="Executing migration" id="create login attempt table" 2024-09-11T07:31:16.195 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.099314917Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-11T07:31:16.195 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.102126913Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-11T07:31:16.195 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.104899711Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-11T07:31:16.195 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.107490914Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-11T07:31:16.195 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.107900067Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-11T07:31:16.195 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.110662141Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-11T07:31:16.195 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.113725415Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-11T07:31:16.195 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.116430989Z level=info msg="Executing migration" id="create user auth table" 2024-09-11T07:31:16.195 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.116845838Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-11T07:31:16.195 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.119862217Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-11T07:31:16.195 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.122564539Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-11T07:31:16.195 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.12543234Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-11T07:31:16.196 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.126229467Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-11T07:31:16.196 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.129036756Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-11T07:31:16.196 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.131815754Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-11T07:31:16.196 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.134765249Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-11T07:31:16.196 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.13724077Z level=info msg="Executing migration" id="create server_lock table" 2024-09-11T07:31:16.196 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.140107547Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-11T07:31:16.196 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.142726416Z level=info msg="Executing migration" id="create user auth token table" 2024-09-11T07:31:16.196 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.145489179Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-11T07:31:16.196 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.14595256Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-11T07:31:16.196 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.148685411Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-11T07:31:16.196 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.15122568Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-11T07:31:16.196 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.153873271Z level=info msg="Executing migration" id="create cache_data table" 2024-09-11T07:31:16.196 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.154310618Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-11T07:31:16.197 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.157058796Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-11T07:31:16.197 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.15979612Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-11T07:31:16.197 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.16247444Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-11T07:31:16.197 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.162803144Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-11T07:31:16.197 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.165537674Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-11T07:31:16.197 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.168413321Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-11T07:31:16.197 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.170989555Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-11T07:31:16.197 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.17147975Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-11T07:31:16.197 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.17241504Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-11T07:31:16.197 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.17314742Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-11T07:31:16.197 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.173572962Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-11T07:31:16.197 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.174419251Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-11T07:31:16.197 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.174864967Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-11T07:31:16.197 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.175654463Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-11T07:31:16.198 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.176529672Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-11T07:31:16.198 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.177306571Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-11T07:31:16.198 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.177746143Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-11T07:31:16.198 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.178693866Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-11T07:31:16.198 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.179420765Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-11T07:31:16.198 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.180248685Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-11T07:31:16.198 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.180718528Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-11T07:31:16.198 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.181143532Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-11T07:31:16.198 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.18195514Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-11T07:31:16.198 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.182395187Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-11T07:31:16.198 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.183199178Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-11T07:31:16.198 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.184079671Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-11T07:31:16.198 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.184516625Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-11T07:31:16.467 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.187940348Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-11T07:31:16.467 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.191495316Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-11T07:31:16.467 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.192362867Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-11T07:31:16.467 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.19279781Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-11T07:31:16.467 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.193683138Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-11T07:31:16.467 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.19451708Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-11T07:31:16.467 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.195367572Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-11T07:31:16.467 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.195907818Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-11T07:31:16.467 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.196773115Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-11T07:31:16.468 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.197630114Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-11T07:31:16.468 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.198518544Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-11T07:31:16.468 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.19936251Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-11T07:31:16.468 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.200226789Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-11T07:31:16.468 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.201186339Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-11T07:31:16.468 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.201911741Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-11T07:31:16.468 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.202748702Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-11T07:31:16.468 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.20580916Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-11T07:31:16.468 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.208507646Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-11T07:31:16.468 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.211264929Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-11T07:31:16.468 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.212234818Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-11T07:31:16.468 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.215252212Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-11T07:31:16.468 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.218000551Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-11T07:31:16.469 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.220756132Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-11T07:31:16.469 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.221189392Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-11T07:31:16.469 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.223923715Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-11T07:31:16.469 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.224816668Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-11T07:31:16.469 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.226682265Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-11T07:31:16.469 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.229750595Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-11T07:31:16.469 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.230768133Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-11T07:31:16.469 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.233822861Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-11T07:31:16.469 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.236605753Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-11T07:31:16.469 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.239357022Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-11T07:31:16.469 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.240292428Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-11T07:31:16.469 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.243138154Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-11T07:31:16.469 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.245882875Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-11T07:31:16.470 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.248382149Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-11T07:31:16.471 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.249315241Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-11T07:31:16.471 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.251880172Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-11T07:31:16.471 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.254760822Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-11T07:31:16.471 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.257363515Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-11T07:31:16.471 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.257873302Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-11T07:31:16.471 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.260557403Z level=info msg="Executing migration" id="create alert_image table" 2024-09-11T07:31:16.471 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.26348295Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-11T07:31:16.471 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.266069108Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-11T07:31:16.471 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.266406845Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-11T07:31:16.471 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.269167786Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-11T07:31:16.471 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.269628125Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T07:31:16.471 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.269700645Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T07:31:16.472 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.272381697Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-11T07:31:16.472 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.272796248Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-11T07:31:16.472 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.275529743Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-11T07:31:16.472 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.275696237Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-11T07:31:16.472 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.275809278Z level=info msg="alerts found to migrate" alerts=0 2024-09-11T07:31:16.472 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.276246211Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-11T07:31:16.472 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.277053775Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-11T07:31:16.472 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.277512608Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-11T07:31:16.472 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.278350227Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-11T07:31:16.472 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.279132258Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-11T07:31:16.472 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.279955039Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-11T07:31:16.472 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.28028869Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-11T07:31:16.472 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.281180562Z level=info msg="Executing migration" id="create data_keys table" 2024-09-11T07:31:16.472 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.281799827Z level=info msg="Executing migration" id="create secrets table" 2024-09-11T07:31:16.473 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.282226629Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-11T07:31:16.473 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.286463873Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-11T07:31:16.473 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.287383208Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-11T07:31:16.473 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.288208042Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-11T07:31:16.473 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.292444594Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-11T07:31:16.473 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.296658956Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-11T07:31:16.473 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.297072368Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-11T07:31:16.473 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.297861616Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-11T07:31:16.473 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.298591574Z level=info msg="Executing migration" id="create permission table" 2024-09-11T07:31:16.473 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.29901185Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-11T07:31:16.473 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.299444831Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-11T07:31:16.473 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.3002549Z level=info msg="Executing migration" id="create role table" 2024-09-11T07:31:16.473 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.301123793Z level=info msg="Executing migration" id="add column display_name" 2024-09-11T07:31:16.474 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.302121648Z level=info msg="Executing migration" id="add column group_name" 2024-09-11T07:31:16.474 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.303116683Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-11T07:31:16.474 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.303886756Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-11T07:31:16.474 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.30489003Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-11T07:31:16.474 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.305577346Z level=info msg="Executing migration" id="create team role table" 2024-09-11T07:31:16.474 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.306056419Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-11T07:31:16.474 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.306915253Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-11T07:31:16.474 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.307905728Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-11T07:31:16.474 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.308579688Z level=info msg="Executing migration" id="create user role table" 2024-09-11T07:31:16.474 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.308989669Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-11T07:31:16.474 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.309990624Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-11T07:31:16.474 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.310794925Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-11T07:31:16.474 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.311758954Z level=info msg="Executing migration" id="create builtin role table" 2024-09-11T07:31:16.474 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.312387574Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-11T07:31:16.475 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.315129842Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-11T07:31:16.475 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.317954026Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-11T07:31:16.475 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.320914546Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-11T07:31:16.475 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.3214235Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-11T07:31:16.475 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.324210044Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-11T07:31:16.475 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.324806995Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-11T07:31:16.475 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.325266966Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-11T07:31:16.475 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.325671939Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-11T07:31:16.475 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.326139475Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-11T07:31:16.475 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.327125266Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-11T07:31:16.475 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.327542383Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-11T07:31:16.475 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.328005316Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-11T07:31:16.475 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.328307428Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-11T07:31:16.475 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.328906442Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-11T07:31:16.476 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.32937699Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-11T07:31:16.476 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.329741317Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-11T07:31:16.476 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.330067646Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-11T07:31:16.476 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.330415509Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-11T07:31:16.476 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.33083086Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-11T07:31:16.476 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.331281774Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-11T07:31:16.476 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.332263791Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-11T07:31:16.476 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.332594187Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-11T07:31:16.476 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.333034694Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-11T07:31:16.476 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.333489283Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-11T07:31:16.476 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.333943485Z level=info msg="Executing migration" id="add correlation config column" 2024-09-11T07:31:16.476 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.33492939Z level=info msg="Executing migration" id="create entity_events table" 2024-09-11T07:31:16.476 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.335362473Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-11T07:31:16.477 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.338080211Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T07:31:16.477 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.338189799Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T07:31:16.477 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.338482738Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T07:31:16.477 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.338565526Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T07:31:16.477 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.338858049Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-11T07:31:16.477 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.341565364Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-11T07:31:16.477 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.34439474Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-11T07:31:16.477 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.347305622Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T07:31:16.477 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.347804835Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-11T07:31:16.477 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.350734296Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T07:31:16.477 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.353551271Z level=info msg="Executing migration" id="Drop public config table" 2024-09-11T07:31:16.477 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.356534163Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-11T07:31:16.477 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.357077487Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-11T07:31:16.478 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.359736329Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T07:31:16.478 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.360307178Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-11T07:31:16.478 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.362558103Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-11T07:31:16.478 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.365676138Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-11T07:31:16.478 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.368502716Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-11T07:31:16.478 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.371063218Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-11T07:31:16.478 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.37385819Z level=info msg="Executing migration" id="add share column" 2024-09-11T07:31:16.478 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.376504Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-11T07:31:16.478 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.376961317Z level=info msg="Executing migration" id="create file table" 2024-09-11T07:31:16.478 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.379800231Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-11T07:31:16.478 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.38234229Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-11T07:31:16.478 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.382791336Z level=info msg="Executing migration" id="create file_meta table" 2024-09-11T07:31:16.478 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.385130465Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-11T07:31:16.478 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.387641524Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-11T07:31:16.479 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.390469922Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-11T07:31:16.479 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.393012154Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-11T07:31:16.479 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.393357186Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-11T07:31:16.479 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.396221582Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-11T07:31:16.479 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.397400833Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-11T07:31:16.479 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.399030237Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-11T07:31:16.479 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.401529449Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-11T07:31:16.479 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.401965085Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-11T07:31:16.479 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.402903095Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-11T07:31:16.479 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.403656235Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-11T07:31:16.479 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.404687582Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-11T07:31:16.479 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.405746022Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-11T07:31:16.479 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.4065814Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-11T07:31:16.479 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.416082503Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-11T07:31:16.480 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.416895601Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-11T07:31:16.480 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.417343145Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-11T07:31:16.480 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.420769713Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-11T07:31:16.480 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.421510997Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-11T07:31:16.480 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.422363858Z level=info msg="Executing migration" id="create folder table" 2024-09-11T07:31:16.480 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.422788917Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-11T07:31:16.480 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.423796213Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-11T07:31:16.480 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.424478211Z level=info msg="Executing migration" id="Update folder title length" 2024-09-11T07:31:16.480 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.42530829Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-11T07:31:16.480 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=migrator t=2024-09-11T07:31:16.425828016Z level=info msg="migrations completed" performed=485 skipped=0 duration=658.287871ms 2024-09-11T07:31:16.480 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=sqlstore t=2024-09-11T07:31:16.426201721Z level=info msg="Created default organization" 2024-09-11T07:31:16.721 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-11T07:31:14.006115+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":929,"num_read_kb":786,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385440,"kb_used_data":6604,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575776,"statfs":{"total":767960285184,"available":765517594624,"internally_reserved":0,"allocated":6762496,"data_stored":3649554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002832"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310542+0000","last_change":"2024-09-11T07:29:55.331068+0000","last_active":"2024-09-11T07:29:59.310542+0000","last_peered":"2024-09-11T07:29:59.310542+0000","last_clean":"2024-09-11T07:29:59.310542+0000","last_became_active":"2024-09-11T07:29:55.330747+0000","last_became_peered":"2024-09-11T07:29:55.330747+0000","last_unstale":"2024-09-11T07:29:59.310542+0000","last_undegraded":"2024-09-11T07:29:59.310542+0000","last_fullsized":"2024-09-11T07:29:59.310542+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:12.694045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192388+0000","last_change":"2024-09-11T07:29:49.220298+0000","last_active":"2024-09-11T07:29:59.192388+0000","last_peered":"2024-09-11T07:29:59.192388+0000","last_clean":"2024-09-11T07:29:59.192388+0000","last_became_active":"2024-09-11T07:29:49.220032+0000","last_became_peered":"2024-09-11T07:29:49.220032+0000","last_unstale":"2024-09-11T07:29:59.192388+0000","last_undegraded":"2024-09-11T07:29:59.192388+0000","last_fullsized":"2024-09-11T07:29:59.192388+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:51:23.777665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.309638+0000","last_change":"2024-09-11T07:29:51.232066+0000","last_active":"2024-09-11T07:29:59.309638+0000","last_peered":"2024-09-11T07:29:59.309638+0000","last_clean":"2024-09-11T07:29:59.309638+0000","last_became_active":"2024-09-11T07:29:51.231779+0000","last_became_peered":"2024-09-11T07:29:51.231779+0000","last_unstale":"2024-09-11T07:29:59.309638+0000","last_undegraded":"2024-09-11T07:29:59.309638+0000","last_fullsized":"2024-09-11T07:29:59.309638+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:03.941638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185511+0000","last_change":"2024-09-11T07:29:53.244513+0000","last_active":"2024-09-11T07:29:59.185511+0000","last_peered":"2024-09-11T07:29:59.185511+0000","last_clean":"2024-09-11T07:29:59.185511+0000","last_became_active":"2024-09-11T07:29:53.244224+0000","last_became_peered":"2024-09-11T07:29:53.244224+0000","last_unstale":"2024-09-11T07:29:59.185511+0000","last_undegraded":"2024-09-11T07:29:59.185511+0000","last_fullsized":"2024-09-11T07:29:59.185511+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:47.625796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310995+0000","last_change":"2024-09-11T07:29:49.219858+0000","last_active":"2024-09-11T07:29:59.310995+0000","last_peered":"2024-09-11T07:29:59.310995+0000","last_clean":"2024-09-11T07:29:59.310995+0000","last_became_active":"2024-09-11T07:29:49.219556+0000","last_became_peered":"2024-09-11T07:29:49.219556+0000","last_unstale":"2024-09-11T07:29:59.310995+0000","last_undegraded":"2024-09-11T07:29:59.310995+0000","last_fullsized":"2024-09-11T07:29:59.310995+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:28:03.841375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185515+0000","last_change":"2024-09-11T07:29:51.228985+0000","last_active":"2024-09-11T07:29:59.185515+0000","last_peered":"2024-09-11T07:29:59.185515+0000","last_clean":"2024-09-11T07:29:59.185515+0000","last_became_active":"2024-09-11T07:29:51.228664+0000","last_became_peered":"2024-09-11T07:29:51.228664+0000","last_unstale":"2024-09-11T07:29:59.185515+0000","last_undegraded":"2024-09-11T07:29:59.185515+0000","last_fullsized":"2024-09-11T07:29:59.185515+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:23:40.462184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.190361+0000","last_change":"2024-09-11T07:29:53.240750+0000","last_active":"2024-09-11T07:29:59.190361+0000","last_peered":"2024-09-11T07:29:59.190361+0000","last_clean":"2024-09-11T07:29:59.190361+0000","last_became_active":"2024-09-11T07:29:53.238382+0000","last_became_peered":"2024-09-11T07:29:53.238382+0000","last_unstale":"2024-09-11T07:29:59.190361+0000","last_undegraded":"2024-09-11T07:29:59.190361+0000","last_fullsized":"2024-09-11T07:29:59.190361+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:45:53.512472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185692+0000","last_change":"2024-09-11T07:29:55.263334+0000","last_active":"2024-09-11T07:29:59.185692+0000","last_peered":"2024-09-11T07:29:59.185692+0000","last_clean":"2024-09-11T07:29:59.185692+0000","last_became_active":"2024-09-11T07:29:55.262985+0000","last_became_peered":"2024-09-11T07:29:55.262985+0000","last_unstale":"2024-09-11T07:29:59.185692+0000","last_undegraded":"2024-09-11T07:29:59.185692+0000","last_fullsized":"2024-09-11T07:29:59.185692+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:27:22.249161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427805+0000","last_change":"2024-09-11T07:29:49.219066+0000","last_active":"2024-09-11T07:29:59.427805+0000","last_peered":"2024-09-11T07:29:59.427805+0000","last_clean":"2024-09-11T07:29:59.427805+0000","last_became_active":"2024-09-11T07:29:49.218738+0000","last_became_peered":"2024-09-11T07:29:49.218738+0000","last_unstale":"2024-09-11T07:29:59.427805+0000","last_undegraded":"2024-09-11T07:29:59.427805+0000","last_fullsized":"2024-09-11T07:29:59.427805+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:01.493359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"60'5","reported_seq":26,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185806+0000","last_change":"2024-09-11T07:29:51.229295+0000","last_active":"2024-09-11T07:29:59.185806+0000","last_peered":"2024-09-11T07:29:59.185806+0000","last_clean":"2024-09-11T07:29:59.185806+0000","last_became_active":"2024-09-11T07:29:51.228974+0000","last_became_peered":"2024-09-11T07:29:51.228974+0000","last_unstale":"2024-09-11T07:29:59.185806+0000","last_undegraded":"2024-09-11T07:29:59.185806+0000","last_fullsized":"2024-09-11T07:29:59.185806+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:59.284205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427471+0000","last_change":"2024-09-11T07:29:53.244230+0000","last_active":"2024-09-11T07:29:59.427471+0000","last_peered":"2024-09-11T07:29:59.427471+0000","last_clean":"2024-09-11T07:29:59.427471+0000","last_became_active":"2024-09-11T07:29:53.243957+0000","last_became_peered":"2024-09-11T07:29:53.243957+0000","last_unstale":"2024-09-11T07:29:59.427471+0000","last_undegraded":"2024-09-11T07:29:59.427471+0000","last_fullsized":"2024-09-11T07:29:59.427471+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:47:06.409144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013189+0000","last_change":"2024-09-11T07:29:55.264649+0000","last_active":"2024-09-11T07:30:00.013189+0000","last_peered":"2024-09-11T07:30:00.013189+0000","last_clean":"2024-09-11T07:30:00.013189+0000","last_became_active":"2024-09-11T07:29:55.264376+0000","last_became_peered":"2024-09-11T07:29:55.264376+0000","last_unstale":"2024-09-11T07:30:00.013189+0000","last_undegraded":"2024-09-11T07:30:00.013189+0000","last_fullsized":"2024-09-11T07:30:00.013189+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:27:18.040554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.428037+0000","last_change":"2024-09-11T07:29:49.221507+0000","last_active":"2024-09-11T07:29:59.428037+0000","last_peered":"2024-09-11T07:29:59.428037+0000","last_clean":"2024-09-11T07:29:59.428037+0000","last_became_active":"2024-09-11T07:29:49.221264+0000","last_became_peered":"2024-09-11T07:29:49.221264+0000","last_unstale":"2024-09-11T07:29:59.428037+0000","last_undegraded":"2024-09-11T07:29:59.428037+0000","last_fullsized":"2024-09-11T07:29:59.428037+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:14:32.086856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185820+0000","last_change":"2024-09-11T07:29:51.229153+0000","last_active":"2024-09-11T07:29:59.185820+0000","last_peered":"2024-09-11T07:29:59.185820+0000","last_clean":"2024-09-11T07:29:59.185820+0000","last_became_active":"2024-09-11T07:29:51.228867+0000","last_became_peered":"2024-09-11T07:29:51.228867+0000","last_unstale":"2024-09-11T07:29:59.185820+0000","last_undegraded":"2024-09-11T07:29:59.185820+0000","last_fullsized":"2024-09-11T07:29:59.185820+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:44:15.343759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013288+0000","last_change":"2024-09-11T07:29:53.247558+0000","last_active":"2024-09-11T07:30:00.013288+0000","last_peered":"2024-09-11T07:30:00.013288+0000","last_clean":"2024-09-11T07:30:00.013288+0000","last_became_active":"2024-09-11T07:29:53.247227+0000","last_became_peered":"2024-09-11T07:29:53.247227+0000","last_unstale":"2024-09-11T07:30:00.013288+0000","last_undegraded":"2024-09-11T07:30:00.013288+0000","last_fullsized":"2024-09-11T07:30:00.013288+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:13.426173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191157+0000","last_change":"2024-09-11T07:29:55.267569+0000","last_active":"2024-09-11T07:29:59.191157+0000","last_peered":"2024-09-11T07:29:59.191157+0000","last_clean":"2024-09-11T07:29:59.191157+0000","last_became_active":"2024-09-11T07:29:55.267262+0000","last_became_peered":"2024-09-11T07:29:55.267262+0000","last_unstale":"2024-09-11T07:29:59.191157+0000","last_undegraded":"2024-09-11T07:29:59.191157+0000","last_fullsized":"2024-09-11T07:29:59.191157+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:27:01.693078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"60'17","reported_seq":44,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.186191+0000","last_change":"2024-09-11T07:29:51.229845+0000","last_active":"2024-09-11T07:29:59.186191+0000","last_peered":"2024-09-11T07:29:59.186191+0000","last_clean":"2024-09-11T07:29:59.186191+0000","last_became_active":"2024-09-11T07:29:51.229452+0000","last_became_peered":"2024-09-11T07:29:51.229452+0000","last_unstale":"2024-09-11T07:29:59.186191+0000","last_undegraded":"2024-09-11T07:29:59.186191+0000","last_fullsized":"2024-09-11T07:29:59.186191+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:12.853243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427887+0000","last_change":"2024-09-11T07:29:49.218197+0000","last_active":"2024-09-11T07:29:59.427887+0000","last_peered":"2024-09-11T07:29:59.427887+0000","last_clean":"2024-09-11T07:29:59.427887+0000","last_became_active":"2024-09-11T07:29:49.217928+0000","last_became_peered":"2024-09-11T07:29:49.217928+0000","last_unstale":"2024-09-11T07:29:59.427887+0000","last_undegraded":"2024-09-11T07:29:59.427887+0000","last_fullsized":"2024-09-11T07:29:59.427887+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:59:21.252858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191368+0000","last_change":"2024-09-11T07:29:53.248486+0000","last_active":"2024-09-11T07:29:59.191368+0000","last_peered":"2024-09-11T07:29:59.191368+0000","last_clean":"2024-09-11T07:29:59.191368+0000","last_became_active":"2024-09-11T07:29:53.248194+0000","last_became_peered":"2024-09-11T07:29:53.248194+0000","last_unstale":"2024-09-11T07:29:59.191368+0000","last_undegraded":"2024-09-11T07:29:59.191368+0000","last_fullsized":"2024-09-11T07:29:59.191368+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:26:20.557318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.601180+0000","last_change":"2024-09-11T07:29:55.266503+0000","last_active":"2024-09-11T07:29:59.601180+0000","last_peered":"2024-09-11T07:29:59.601180+0000","last_clean":"2024-09-11T07:29:59.601180+0000","last_became_active":"2024-09-11T07:29:55.266240+0000","last_became_peered":"2024-09-11T07:29:55.266240+0000","last_unstale":"2024-09-11T07:29:59.601180+0000","last_undegraded":"2024-09-11T07:29:59.601180+0000","last_fullsized":"2024-09-11T07:29:59.601180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:34:37.890636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185990+0000","last_change":"2024-09-11T07:29:51.228113+0000","last_active":"2024-09-11T07:29:59.185990+0000","last_peered":"2024-09-11T07:29:59.185990+0000","last_clean":"2024-09-11T07:29:59.185990+0000","last_became_active":"2024-09-11T07:29:51.227100+0000","last_became_peered":"2024-09-11T07:29:51.227100+0000","last_unstale":"2024-09-11T07:29:59.185990+0000","last_undegraded":"2024-09-11T07:29:59.185990+0000","last_fullsized":"2024-09-11T07:29:59.185990+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:38.948094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191086+0000","last_change":"2024-09-11T07:29:49.224400+0000","last_active":"2024-09-11T07:29:59.191086+0000","last_peered":"2024-09-11T07:29:59.191086+0000","last_clean":"2024-09-11T07:29:59.191086+0000","last_became_active":"2024-09-11T07:29:49.224025+0000","last_became_peered":"2024-09-11T07:29:49.224025+0000","last_unstale":"2024-09-11T07:29:59.191086+0000","last_undegraded":"2024-09-11T07:29:59.191086+0000","last_fullsized":"2024-09-11T07:29:59.191086+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:50:09.460643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"60'8","reported_seq":41,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457352+0000","last_change":"2024-09-11T07:29:53.240946+0000","last_active":"2024-09-11T07:31:02.457352+0000","last_peered":"2024-09-11T07:31:02.457352+0000","last_clean":"2024-09-11T07:31:02.457352+0000","last_became_active":"2024-09-11T07:29:53.240648+0000","last_became_peered":"2024-09-11T07:29:53.240648+0000","last_unstale":"2024-09-11T07:31:02.457352+0000","last_undegraded":"2024-09-11T07:31:02.457352+0000","last_fullsized":"2024-09-11T07:31:02.457352+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:04.275293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.186055+0000","last_change":"2024-09-11T07:29:55.267221+0000","last_active":"2024-09-11T07:29:59.186055+0000","last_peered":"2024-09-11T07:29:59.186055+0000","last_clean":"2024-09-11T07:29:59.186055+0000","last_became_active":"2024-09-11T07:29:55.266973+0000","last_became_peered":"2024-09-11T07:29:55.266973+0000","last_unstale":"2024-09-11T07:29:59.186055+0000","last_undegraded":"2024-09-11T07:29:59.186055+0000","last_fullsized":"2024-09-11T07:29:59.186055+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:00:11.369689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191847+0000","last_change":"2024-09-11T07:29:51.230533+0000","last_active":"2024-09-11T07:29:59.191847+0000","last_peered":"2024-09-11T07:29:59.191847+0000","last_clean":"2024-09-11T07:29:59.191847+0000","last_became_active":"2024-09-11T07:29:51.230207+0000","last_became_peered":"2024-09-11T07:29:51.230207+0000","last_unstale":"2024-09-11T07:29:59.191847+0000","last_undegraded":"2024-09-11T07:29:59.191847+0000","last_fullsized":"2024-09-11T07:29:59.191847+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:44.786524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.190536+0000","last_change":"2024-09-11T07:29:49.219305+0000","last_active":"2024-09-11T07:29:59.190536+0000","last_peered":"2024-09-11T07:29:59.190536+0000","last_clean":"2024-09-11T07:29:59.190536+0000","last_became_active":"2024-09-11T07:29:49.219056+0000","last_became_peered":"2024-09-11T07:29:49.219056+0000","last_unstale":"2024-09-11T07:29:59.190536+0000","last_undegraded":"2024-09-11T07:29:59.190536+0000","last_fullsized":"2024-09-11T07:29:59.190536+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:49:21.152009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457459+0000","last_change":"2024-09-11T07:29:53.246435+0000","last_active":"2024-09-11T07:31:02.457459+0000","last_peered":"2024-09-11T07:31:02.457459+0000","last_clean":"2024-09-11T07:31:02.457459+0000","last_became_active":"2024-09-11T07:29:53.246150+0000","last_became_peered":"2024-09-11T07:29:53.246150+0000","last_unstale":"2024-09-11T07:31:02.457459+0000","last_undegraded":"2024-09-11T07:31:02.457459+0000","last_fullsized":"2024-09-11T07:31:02.457459+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:39:34.124732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013513+0000","last_change":"2024-09-11T07:29:55.264617+0000","last_active":"2024-09-11T07:30:00.013513+0000","last_peered":"2024-09-11T07:30:00.013513+0000","last_clean":"2024-09-11T07:30:00.013513+0000","last_became_active":"2024-09-11T07:29:55.264321+0000","last_became_peered":"2024-09-11T07:29:55.264321+0000","last_unstale":"2024-09-11T07:30:00.013513+0000","last_undegraded":"2024-09-11T07:30:00.013513+0000","last_fullsized":"2024-09-11T07:30:00.013513+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:34:41.600998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.186191+0000","last_change":"2024-09-11T07:29:51.227712+0000","last_active":"2024-09-11T07:29:59.186191+0000","last_peered":"2024-09-11T07:29:59.186191+0000","last_clean":"2024-09-11T07:29:59.186191+0000","last_became_active":"2024-09-11T07:29:51.227366+0000","last_became_peered":"2024-09-11T07:29:51.227366+0000","last_unstale":"2024-09-11T07:29:59.186191+0000","last_undegraded":"2024-09-11T07:29:59.186191+0000","last_fullsized":"2024-09-11T07:29:59.186191+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:32.940047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427512+0000","last_change":"2024-09-11T07:29:49.220244+0000","last_active":"2024-09-11T07:29:59.427512+0000","last_peered":"2024-09-11T07:29:59.427512+0000","last_clean":"2024-09-11T07:29:59.427512+0000","last_became_active":"2024-09-11T07:29:49.219961+0000","last_became_peered":"2024-09-11T07:29:49.219961+0000","last_unstale":"2024-09-11T07:29:59.427512+0000","last_undegraded":"2024-09-11T07:29:59.427512+0000","last_fullsized":"2024-09-11T07:29:59.427512+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:50.888550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013648+0000","last_change":"2024-09-11T07:29:53.242774+0000","last_active":"2024-09-11T07:30:00.013648+0000","last_peered":"2024-09-11T07:30:00.013648+0000","last_clean":"2024-09-11T07:30:00.013648+0000","last_became_active":"2024-09-11T07:29:53.242484+0000","last_became_peered":"2024-09-11T07:29:53.242484+0000","last_unstale":"2024-09-11T07:30:00.013648+0000","last_undegraded":"2024-09-11T07:30:00.013648+0000","last_fullsized":"2024-09-11T07:30:00.013648+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:37:23.956005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310730+0000","last_change":"2024-09-11T07:29:55.328486+0000","last_active":"2024-09-11T07:29:59.310730+0000","last_peered":"2024-09-11T07:29:59.310730+0000","last_clean":"2024-09-11T07:29:59.310730+0000","last_became_active":"2024-09-11T07:29:55.328172+0000","last_became_peered":"2024-09-11T07:29:55.328172+0000","last_unstale":"2024-09-11T07:29:59.310730+0000","last_undegraded":"2024-09-11T07:29:59.310730+0000","last_fullsized":"2024-09-11T07:29:59.310730+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:03.124123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"60'14","reported_seq":37,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185642+0000","last_change":"2024-09-11T07:29:51.223754+0000","last_active":"2024-09-11T07:29:59.185642+0000","last_peered":"2024-09-11T07:29:59.185642+0000","last_clean":"2024-09-11T07:29:59.185642+0000","last_became_active":"2024-09-11T07:29:51.223386+0000","last_became_peered":"2024-09-11T07:29:51.223386+0000","last_unstale":"2024-09-11T07:29:59.185642+0000","last_undegraded":"2024-09-11T07:29:59.185642+0000","last_fullsized":"2024-09-11T07:29:59.185642+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:42:42.242532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.575134+0000","last_change":"2024-09-11T07:29:49.217087+0000","last_active":"2024-09-11T07:29:59.575134+0000","last_peered":"2024-09-11T07:29:59.575134+0000","last_clean":"2024-09-11T07:29:59.575134+0000","last_became_active":"2024-09-11T07:29:49.216808+0000","last_became_peered":"2024-09-11T07:29:49.216808+0000","last_unstale":"2024-09-11T07:29:59.575134+0000","last_undegraded":"2024-09-11T07:29:59.575134+0000","last_fullsized":"2024-09-11T07:29:59.575134+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:12.674171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457322+0000","last_change":"2024-09-11T07:29:53.252350+0000","last_active":"2024-09-11T07:31:02.457322+0000","last_peered":"2024-09-11T07:31:02.457322+0000","last_clean":"2024-09-11T07:31:02.457322+0000","last_became_active":"2024-09-11T07:29:53.251440+0000","last_became_peered":"2024-09-11T07:29:53.251440+0000","last_unstale":"2024-09-11T07:31:02.457322+0000","last_undegraded":"2024-09-11T07:31:02.457322+0000","last_fullsized":"2024-09-11T07:31:02.457322+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:24:14.998472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427652+0000","last_change":"2024-09-11T07:29:55.330660+0000","last_active":"2024-09-11T07:29:59.427652+0000","last_peered":"2024-09-11T07:29:59.427652+0000","last_clean":"2024-09-11T07:29:59.427652+0000","last_became_active":"2024-09-11T07:29:55.330307+0000","last_became_peered":"2024-09-11T07:29:55.330307+0000","last_unstale":"2024-09-11T07:29:59.427652+0000","last_undegraded":"2024-09-11T07:29:59.427652+0000","last_fullsized":"2024-09-11T07:29:59.427652+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:10:44.928896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"60'18","reported_seq":48,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.309549+0000","last_change":"2024-09-11T07:29:51.249652+0000","last_active":"2024-09-11T07:29:59.309549+0000","last_peered":"2024-09-11T07:29:59.309549+0000","last_clean":"2024-09-11T07:29:59.309549+0000","last_became_active":"2024-09-11T07:29:51.249369+0000","last_became_peered":"2024-09-11T07:29:51.249369+0000","last_unstale":"2024-09-11T07:29:59.309549+0000","last_undegraded":"2024-09-11T07:29:59.309549+0000","last_fullsized":"2024-09-11T07:29:59.309549+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:34:41.806338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191847+0000","last_change":"2024-09-11T07:29:49.212730+0000","last_active":"2024-09-11T07:29:59.191847+0000","last_peered":"2024-09-11T07:29:59.191847+0000","last_clean":"2024-09-11T07:29:59.191847+0000","last_became_active":"2024-09-11T07:29:49.212431+0000","last_became_peered":"2024-09-11T07:29:49.212431+0000","last_unstale":"2024-09-11T07:29:59.191847+0000","last_undegraded":"2024-09-11T07:29:59.191847+0000","last_fullsized":"2024-09-11T07:29:59.191847+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:49:59.391366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185565+0000","last_change":"2024-09-11T07:29:53.249204+0000","last_active":"2024-09-11T07:29:59.185565+0000","last_peered":"2024-09-11T07:29:59.185565+0000","last_clean":"2024-09-11T07:29:59.185565+0000","last_became_active":"2024-09-11T07:29:53.248928+0000","last_became_peered":"2024-09-11T07:29:53.248928+0000","last_unstale":"2024-09-11T07:29:59.185565+0000","last_undegraded":"2024-09-11T07:29:59.185565+0000","last_fullsized":"2024-09-11T07:29:59.185565+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:59:07.692517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185793+0000","last_change":"2024-09-11T07:29:55.258609+0000","last_active":"2024-09-11T07:29:59.185793+0000","last_peered":"2024-09-11T07:29:59.185793+0000","last_clean":"2024-09-11T07:29:59.185793+0000","last_became_active":"2024-09-11T07:29:55.258346+0000","last_became_peered":"2024-09-11T07:29:55.258346+0000","last_unstale":"2024-09-11T07:29:59.185793+0000","last_undegraded":"2024-09-11T07:29:59.185793+0000","last_fullsized":"2024-09-11T07:29:59.185793+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:48:30.919282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"60'19","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192456+0000","last_change":"2024-09-11T07:29:51.249559+0000","last_active":"2024-09-11T07:29:59.192456+0000","last_peered":"2024-09-11T07:29:59.192456+0000","last_clean":"2024-09-11T07:29:59.192456+0000","last_became_active":"2024-09-11T07:29:51.249303+0000","last_became_peered":"2024-09-11T07:29:51.249303+0000","last_unstale":"2024-09-11T07:29:59.192456+0000","last_undegraded":"2024-09-11T07:29:59.192456+0000","last_fullsized":"2024-09-11T07:29:59.192456+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:10:20.345316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427894+0000","last_change":"2024-09-11T07:29:49.222049+0000","last_active":"2024-09-11T07:29:59.427894+0000","last_peered":"2024-09-11T07:29:59.427894+0000","last_clean":"2024-09-11T07:29:59.427894+0000","last_became_active":"2024-09-11T07:29:49.221775+0000","last_became_peered":"2024-09-11T07:29:49.221775+0000","last_unstale":"2024-09-11T07:29:59.427894+0000","last_undegraded":"2024-09-11T07:29:59.427894+0000","last_fullsized":"2024-09-11T07:29:59.427894+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:34:45.527666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.574531+0000","last_change":"2024-09-11T07:29:53.248215+0000","last_active":"2024-09-11T07:29:59.574531+0000","last_peered":"2024-09-11T07:29:59.574531+0000","last_clean":"2024-09-11T07:29:59.574531+0000","last_became_active":"2024-09-11T07:29:53.247969+0000","last_became_peered":"2024-09-11T07:29:53.247969+0000","last_unstale":"2024-09-11T07:29:59.574531+0000","last_undegraded":"2024-09-11T07:29:59.574531+0000","last_fullsized":"2024-09-11T07:29:59.574531+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:42.535405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.190345+0000","last_change":"2024-09-11T07:29:55.328531+0000","last_active":"2024-09-11T07:29:59.190345+0000","last_peered":"2024-09-11T07:29:59.190345+0000","last_clean":"2024-09-11T07:29:59.190345+0000","last_became_active":"2024-09-11T07:29:55.328219+0000","last_became_peered":"2024-09-11T07:29:55.328219+0000","last_unstale":"2024-09-11T07:29:59.190345+0000","last_undegraded":"2024-09-11T07:29:59.190345+0000","last_fullsized":"2024-09-11T07:29:59.190345+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:14:43.652813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.190876+0000","last_change":"2024-09-11T07:29:51.230569+0000","last_active":"2024-09-11T07:29:59.190876+0000","last_peered":"2024-09-11T07:29:59.190876+0000","last_clean":"2024-09-11T07:29:59.190876+0000","last_became_active":"2024-09-11T07:29:51.230279+0000","last_became_peered":"2024-09-11T07:29:51.230279+0000","last_unstale":"2024-09-11T07:29:59.190876+0000","last_undegraded":"2024-09-11T07:29:59.190876+0000","last_fullsized":"2024-09-11T07:29:59.190876+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:04.053471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192064+0000","last_change":"2024-09-11T07:29:49.217145+0000","last_active":"2024-09-11T07:29:59.192064+0000","last_peered":"2024-09-11T07:29:59.192064+0000","last_clean":"2024-09-11T07:29:59.192064+0000","last_became_active":"2024-09-11T07:29:49.216859+0000","last_became_peered":"2024-09-11T07:29:49.216859+0000","last_unstale":"2024-09-11T07:29:59.192064+0000","last_undegraded":"2024-09-11T07:29:59.192064+0000","last_fullsized":"2024-09-11T07:29:59.192064+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:44.142846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457320+0000","last_change":"2024-09-11T07:29:53.242824+0000","last_active":"2024-09-11T07:31:02.457320+0000","last_peered":"2024-09-11T07:31:02.457320+0000","last_clean":"2024-09-11T07:31:02.457320+0000","last_became_active":"2024-09-11T07:29:53.242428+0000","last_became_peered":"2024-09-11T07:29:53.242428+0000","last_unstale":"2024-09-11T07:31:02.457320+0000","last_undegraded":"2024-09-11T07:31:02.457320+0000","last_fullsized":"2024-09-11T07:31:02.457320+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:11.714924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191040+0000","last_change":"2024-09-11T07:29:55.264537+0000","last_active":"2024-09-11T07:29:59.191040+0000","last_peered":"2024-09-11T07:29:59.191040+0000","last_clean":"2024-09-11T07:29:59.191040+0000","last_became_active":"2024-09-11T07:29:55.264268+0000","last_became_peered":"2024-09-11T07:29:55.264268+0000","last_unstale":"2024-09-11T07:29:59.191040+0000","last_undegraded":"2024-09-11T07:29:59.191040+0000","last_fullsized":"2024-09-11T07:29:59.191040+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:34:27.037273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"60'28","reported_seq":80,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457352+0000","last_change":"2024-09-11T07:29:51.229849+0000","last_active":"2024-09-11T07:31:02.457352+0000","last_peered":"2024-09-11T07:31:02.457352+0000","last_clean":"2024-09-11T07:31:02.457352+0000","last_became_active":"2024-09-11T07:29:51.229599+0000","last_became_peered":"2024-09-11T07:29:51.229599+0000","last_unstale":"2024-09-11T07:31:02.457352+0000","last_undegraded":"2024-09-11T07:31:02.457352+0000","last_fullsized":"2024-09-11T07:31:02.457352+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:02.005471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013430+0000","last_change":"2024-09-11T07:29:49.212467+0000","last_active":"2024-09-11T07:30:00.013430+0000","last_peered":"2024-09-11T07:30:00.013430+0000","last_clean":"2024-09-11T07:30:00.013430+0000","last_became_active":"2024-09-11T07:29:49.212218+0000","last_became_peered":"2024-09-11T07:29:49.212218+0000","last_unstale":"2024-09-11T07:30:00.013430+0000","last_undegraded":"2024-09-11T07:30:00.013430+0000","last_fullsized":"2024-09-11T07:30:00.013430+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:32:24.863053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191906+0000","last_change":"2024-09-11T07:29:53.252365+0000","last_active":"2024-09-11T07:29:59.191906+0000","last_peered":"2024-09-11T07:29:59.191906+0000","last_clean":"2024-09-11T07:29:59.191906+0000","last_became_active":"2024-09-11T07:29:53.251781+0000","last_became_peered":"2024-09-11T07:29:53.251781+0000","last_unstale":"2024-09-11T07:29:59.191906+0000","last_undegraded":"2024-09-11T07:29:59.191906+0000","last_fullsized":"2024-09-11T07:29:59.191906+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:41:46.205252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310888+0000","last_change":"2024-09-11T07:29:55.257021+0000","last_active":"2024-09-11T07:29:59.310888+0000","last_peered":"2024-09-11T07:29:59.310888+0000","last_clean":"2024-09-11T07:29:59.310888+0000","last_became_active":"2024-09-11T07:29:55.256640+0000","last_became_peered":"2024-09-11T07:29:55.256640+0000","last_unstale":"2024-09-11T07:29:59.310888+0000","last_undegraded":"2024-09-11T07:29:59.310888+0000","last_fullsized":"2024-09-11T07:29:59.310888+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:04:13.829670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"60'13","reported_seq":43,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191956+0000","last_change":"2024-09-11T07:29:51.229071+0000","last_active":"2024-09-11T07:29:59.191956+0000","last_peered":"2024-09-11T07:29:59.191956+0000","last_clean":"2024-09-11T07:29:59.191956+0000","last_became_active":"2024-09-11T07:29:51.228793+0000","last_became_peered":"2024-09-11T07:29:51.228793+0000","last_unstale":"2024-09-11T07:29:59.191956+0000","last_undegraded":"2024-09-11T07:29:59.191956+0000","last_fullsized":"2024-09-11T07:29:59.191956+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:21.473187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"61'5","reported_seq":102,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:10.255165+0000","last_change":"2024-09-11T07:29:53.233096+0000","last_active":"2024-09-11T07:31:10.255165+0000","last_peered":"2024-09-11T07:31:10.255165+0000","last_clean":"2024-09-11T07:31:10.255165+0000","last_became_active":"2024-09-11T07:29:49.212382+0000","last_became_peered":"2024-09-11T07:29:49.212382+0000","last_unstale":"2024-09-11T07:31:10.255165+0000","last_undegraded":"2024-09-11T07:31:10.255165+0000","last_fullsized":"2024-09-11T07:31:10.255165+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:10.898880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059387800000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.600650+0000","last_change":"2024-09-11T07:29:49.218737+0000","last_active":"2024-09-11T07:29:59.600650+0000","last_peered":"2024-09-11T07:29:59.600650+0000","last_clean":"2024-09-11T07:29:59.600650+0000","last_became_active":"2024-09-11T07:29:49.218493+0000","last_became_peered":"2024-09-11T07:29:49.218493+0000","last_unstale":"2024-09-11T07:29:59.600650+0000","last_undegraded":"2024-09-11T07:29:59.600650+0000","last_fullsized":"2024-09-11T07:29:59.600650+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:54:22.783964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.601196+0000","last_change":"2024-09-11T07:29:53.240224+0000","last_active":"2024-09-11T07:29:59.601196+0000","last_peered":"2024-09-11T07:29:59.601196+0000","last_clean":"2024-09-11T07:29:59.601196+0000","last_became_active":"2024-09-11T07:29:53.239937+0000","last_became_peered":"2024-09-11T07:29:53.239937+0000","last_unstale":"2024-09-11T07:29:59.601196+0000","last_undegraded":"2024-09-11T07:29:59.601196+0000","last_fullsized":"2024-09-11T07:29:59.601196+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:49:42.143133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427661+0000","last_change":"2024-09-11T07:29:55.328576+0000","last_active":"2024-09-11T07:29:59.427661+0000","last_peered":"2024-09-11T07:29:59.427661+0000","last_clean":"2024-09-11T07:29:59.427661+0000","last_became_active":"2024-09-11T07:29:55.328292+0000","last_became_peered":"2024-09-11T07:29:55.328292+0000","last_unstale":"2024-09-11T07:29:59.427661+0000","last_undegraded":"2024-09-11T07:29:59.427661+0000","last_fullsized":"2024-09-11T07:29:59.427661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:51.100385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"60'12","reported_seq":34,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192610+0000","last_change":"2024-09-11T07:29:51.233079+0000","last_active":"2024-09-11T07:29:59.192610+0000","last_peered":"2024-09-11T07:29:59.192610+0000","last_clean":"2024-09-11T07:29:59.192610+0000","last_became_active":"2024-09-11T07:29:51.232822+0000","last_became_peered":"2024-09-11T07:29:51.232822+0000","last_unstale":"2024-09-11T07:29:59.192610+0000","last_undegraded":"2024-09-11T07:29:59.192610+0000","last_fullsized":"2024-09-11T07:29:59.192610+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:26.858631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"55'1","reported_seq":24,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191337+0000","last_change":"2024-09-11T07:29:53.241535+0000","last_active":"2024-09-11T07:29:59.191337+0000","last_peered":"2024-09-11T07:29:59.191337+0000","last_clean":"2024-09-11T07:29:59.191337+0000","last_became_active":"2024-09-11T07:29:49.220131+0000","last_became_peered":"2024-09-11T07:29:49.220131+0000","last_unstale":"2024-09-11T07:29:59.191337+0000","last_undegraded":"2024-09-11T07:29:59.191337+0000","last_fullsized":"2024-09-11T07:29:59.191337+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:10.835176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049260989999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427504+0000","last_change":"2024-09-11T07:29:49.220533+0000","last_active":"2024-09-11T07:29:59.427504+0000","last_peered":"2024-09-11T07:29:59.427504+0000","last_clean":"2024-09-11T07:29:59.427504+0000","last_became_active":"2024-09-11T07:29:49.220271+0000","last_became_peered":"2024-09-11T07:29:49.220271+0000","last_unstale":"2024-09-11T07:29:59.427504+0000","last_undegraded":"2024-09-11T07:29:59.427504+0000","last_fullsized":"2024-09-11T07:29:59.427504+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:57.145971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013689+0000","last_change":"2024-09-11T07:29:53.247273+0000","last_active":"2024-09-11T07:30:00.013689+0000","last_peered":"2024-09-11T07:30:00.013689+0000","last_clean":"2024-09-11T07:30:00.013689+0000","last_became_active":"2024-09-11T07:29:53.246977+0000","last_became_peered":"2024-09-11T07:29:53.246977+0000","last_unstale":"2024-09-11T07:30:00.013689+0000","last_undegraded":"2024-09-11T07:30:00.013689+0000","last_fullsized":"2024-09-11T07:30:00.013689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:03:30.499901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191284+0000","last_change":"2024-09-11T07:29:55.264599+0000","last_active":"2024-09-11T07:29:59.191284+0000","last_peered":"2024-09-11T07:29:59.191284+0000","last_clean":"2024-09-11T07:29:59.191284+0000","last_became_active":"2024-09-11T07:29:55.264276+0000","last_became_peered":"2024-09-11T07:29:55.264276+0000","last_unstale":"2024-09-11T07:29:59.191284+0000","last_undegraded":"2024-09-11T07:29:59.191284+0000","last_fullsized":"2024-09-11T07:29:59.191284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:26.370166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"60'14","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.574159+0000","last_change":"2024-09-11T07:29:51.249515+0000","last_active":"2024-09-11T07:29:59.574159+0000","last_peered":"2024-09-11T07:29:59.574159+0000","last_clean":"2024-09-11T07:29:59.574159+0000","last_became_active":"2024-09-11T07:29:51.249236+0000","last_became_peered":"2024-09-11T07:29:51.249236+0000","last_unstale":"2024-09-11T07:29:59.574159+0000","last_undegraded":"2024-09-11T07:29:59.574159+0000","last_fullsized":"2024-09-11T07:29:59.574159+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:00:08.555358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"57'2","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310689+0000","last_change":"2024-09-11T07:29:53.244519+0000","last_active":"2024-09-11T07:29:59.310689+0000","last_peered":"2024-09-11T07:29:59.310689+0000","last_clean":"2024-09-11T07:29:59.310689+0000","last_became_active":"2024-09-11T07:29:49.216312+0000","last_became_peered":"2024-09-11T07:29:49.216312+0000","last_unstale":"2024-09-11T07:29:59.310689+0000","last_undegraded":"2024-09-11T07:29:59.310689+0000","last_fullsized":"2024-09-11T07:29:59.310689+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:09.996156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019793269999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"53'1","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013399+0000","last_change":"2024-09-11T07:29:49.219502+0000","last_active":"2024-09-11T07:30:00.013399+0000","last_peered":"2024-09-11T07:30:00.013399+0000","last_clean":"2024-09-11T07:30:00.013399+0000","last_became_active":"2024-09-11T07:29:49.219246+0000","last_became_peered":"2024-09-11T07:29:49.219246+0000","last_unstale":"2024-09-11T07:30:00.013399+0000","last_undegraded":"2024-09-11T07:30:00.013399+0000","last_fullsized":"2024-09-11T07:30:00.013399+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:09:50.493982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"62'79","reported_seq":52,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:30:28.046381+0000","last_change":"2024-09-11T07:29:38.923543+0000","last_active":"2024-09-11T07:30:28.046381+0000","last_peered":"2024-09-11T07:30:28.046381+0000","last_clean":"2024-09-11T07:30:28.046381+0000","last_became_active":"2024-09-11T07:29:38.917813+0000","last_became_peered":"2024-09-11T07:29:38.917813+0000","last_unstale":"2024-09-11T07:30:28.046381+0000","last_undegraded":"2024-09-11T07:30:28.046381+0000","last_fullsized":"2024-09-11T07:30:28.046381+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.984340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:45.984340+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.984340+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:58:43.411014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427949+0000","last_change":"2024-09-11T07:29:53.240076+0000","last_active":"2024-09-11T07:29:59.427949+0000","last_peered":"2024-09-11T07:29:59.427949+0000","last_clean":"2024-09-11T07:29:59.427949+0000","last_became_active":"2024-09-11T07:29:53.239779+0000","last_became_peered":"2024-09-11T07:29:53.239779+0000","last_unstale":"2024-09-11T07:29:59.427949+0000","last_undegraded":"2024-09-11T07:29:59.427949+0000","last_fullsized":"2024-09-11T07:29:59.427949+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:16.177970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"60'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013735+0000","last_change":"2024-09-11T07:29:55.259465+0000","last_active":"2024-09-11T07:30:00.013735+0000","last_peered":"2024-09-11T07:30:00.013735+0000","last_clean":"2024-09-11T07:30:00.013735+0000","last_became_active":"2024-09-11T07:29:55.259230+0000","last_became_peered":"2024-09-11T07:29:55.259230+0000","last_unstale":"2024-09-11T07:30:00.013735+0000","last_undegraded":"2024-09-11T07:30:00.013735+0000","last_fullsized":"2024-09-11T07:30:00.013735+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:50.889796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"60'19","reported_seq":47,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.574134+0000","last_change":"2024-09-11T07:29:51.249454+0000","last_active":"2024-09-11T07:29:59.574134+0000","last_peered":"2024-09-11T07:29:59.574134+0000","last_clean":"2024-09-11T07:29:59.574134+0000","last_became_active":"2024-09-11T07:29:51.249161+0000","last_became_peered":"2024-09-11T07:29:51.249161+0000","last_unstale":"2024-09-11T07:29:59.574134+0000","last_undegraded":"2024-09-11T07:29:59.574134+0000","last_fullsized":"2024-09-11T07:29:59.574134+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:04.426603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.601279+0000","last_change":"2024-09-11T07:29:49.213494+0000","last_active":"2024-09-11T07:29:59.601279+0000","last_peered":"2024-09-11T07:29:59.601279+0000","last_clean":"2024-09-11T07:29:59.601279+0000","last_became_active":"2024-09-11T07:29:49.213232+0000","last_became_peered":"2024-09-11T07:29:49.213232+0000","last_unstale":"2024-09-11T07:29:59.601279+0000","last_undegraded":"2024-09-11T07:29:59.601279+0000","last_fullsized":"2024-09-11T07:29:59.601279+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:34:40.475670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.600876+0000","last_change":"2024-09-11T07:29:53.247329+0000","last_active":"2024-09-11T07:29:59.600876+0000","last_peered":"2024-09-11T07:29:59.600876+0000","last_clean":"2024-09-11T07:29:59.600876+0000","last_became_active":"2024-09-11T07:29:53.245070+0000","last_became_peered":"2024-09-11T07:29:53.245070+0000","last_unstale":"2024-09-11T07:29:59.600876+0000","last_undegraded":"2024-09-11T07:29:59.600876+0000","last_fullsized":"2024-09-11T07:29:59.600876+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:44:18.389917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427582+0000","last_change":"2024-09-11T07:29:55.261521+0000","last_active":"2024-09-11T07:29:59.427582+0000","last_peered":"2024-09-11T07:29:59.427582+0000","last_clean":"2024-09-11T07:29:59.427582+0000","last_became_active":"2024-09-11T07:29:55.261246+0000","last_became_peered":"2024-09-11T07:29:55.261246+0000","last_unstale":"2024-09-11T07:29:59.427582+0000","last_undegraded":"2024-09-11T07:29:59.427582+0000","last_fullsized":"2024-09-11T07:29:59.427582+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:35.700697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192415+0000","last_change":"2024-09-11T07:29:51.234923+0000","last_active":"2024-09-11T07:29:59.192415+0000","last_peered":"2024-09-11T07:29:59.192415+0000","last_clean":"2024-09-11T07:29:59.192415+0000","last_became_active":"2024-09-11T07:29:51.234559+0000","last_became_peered":"2024-09-11T07:29:51.234559+0000","last_unstale":"2024-09-11T07:29:59.192415+0000","last_undegraded":"2024-09-11T07:29:59.192415+0000","last_fullsized":"2024-09-11T07:29:59.192415+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:53.995040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192193+0000","last_change":"2024-09-11T07:29:49.217242+0000","last_active":"2024-09-11T07:29:59.192193+0000","last_peered":"2024-09-11T07:29:59.192193+0000","last_clean":"2024-09-11T07:29:59.192193+0000","last_became_active":"2024-09-11T07:29:49.216626+0000","last_became_peered":"2024-09-11T07:29:49.216626+0000","last_unstale":"2024-09-11T07:29:59.192193+0000","last_undegraded":"2024-09-11T07:29:59.192193+0000","last_fullsized":"2024-09-11T07:29:59.192193+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:04.040175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.600799+0000","last_change":"2024-09-11T07:29:53.247105+0000","last_active":"2024-09-11T07:29:59.600799+0000","last_peered":"2024-09-11T07:29:59.600799+0000","last_clean":"2024-09-11T07:29:59.600799+0000","last_became_active":"2024-09-11T07:29:53.246297+0000","last_became_peered":"2024-09-11T07:29:53.246297+0000","last_unstale":"2024-09-11T07:29:59.600799+0000","last_undegraded":"2024-09-11T07:29:59.600799+0000","last_fullsized":"2024-09-11T07:29:59.600799+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:03:52.143718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191190+0000","last_change":"2024-09-11T07:29:55.267603+0000","last_active":"2024-09-11T07:29:59.191190+0000","last_peered":"2024-09-11T07:29:59.191190+0000","last_clean":"2024-09-11T07:29:59.191190+0000","last_became_active":"2024-09-11T07:29:55.267294+0000","last_became_peered":"2024-09-11T07:29:55.267294+0000","last_unstale":"2024-09-11T07:29:59.191190+0000","last_undegraded":"2024-09-11T07:29:59.191190+0000","last_fullsized":"2024-09-11T07:29:59.191190+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:01.503521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013396+0000","last_change":"2024-09-11T07:29:51.249934+0000","last_active":"2024-09-11T07:30:00.013396+0000","last_peered":"2024-09-11T07:30:00.013396+0000","last_clean":"2024-09-11T07:30:00.013396+0000","last_became_active":"2024-09-11T07:29:51.249601+0000","last_became_peered":"2024-09-11T07:29:51.249601+0000","last_unstale":"2024-09-11T07:30:00.013396+0000","last_undegraded":"2024-09-11T07:30:00.013396+0000","last_fullsized":"2024-09-11T07:30:00.013396+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:50:25.219576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.601286+0000","last_change":"2024-09-11T07:29:49.219024+0000","last_active":"2024-09-11T07:29:59.601286+0000","last_peered":"2024-09-11T07:29:59.601286+0000","last_clean":"2024-09-11T07:29:59.601286+0000","last_became_active":"2024-09-11T07:29:49.218770+0000","last_became_peered":"2024-09-11T07:29:49.218770+0000","last_unstale":"2024-09-11T07:29:59.601286+0000","last_undegraded":"2024-09-11T07:29:59.601286+0000","last_fullsized":"2024-09-11T07:29:59.601286+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:07:25.270993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457430+0000","last_change":"2024-09-11T07:29:53.246183+0000","last_active":"2024-09-11T07:31:02.457430+0000","last_peered":"2024-09-11T07:31:02.457430+0000","last_clean":"2024-09-11T07:31:02.457430+0000","last_became_active":"2024-09-11T07:29:53.244782+0000","last_became_peered":"2024-09-11T07:29:53.244782+0000","last_unstale":"2024-09-11T07:31:02.457430+0000","last_undegraded":"2024-09-11T07:31:02.457430+0000","last_fullsized":"2024-09-11T07:31:02.457430+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:15.347044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013196+0000","last_change":"2024-09-11T07:29:55.328350+0000","last_active":"2024-09-11T07:30:00.013196+0000","last_peered":"2024-09-11T07:30:00.013196+0000","last_clean":"2024-09-11T07:30:00.013196+0000","last_became_active":"2024-09-11T07:29:55.327990+0000","last_became_peered":"2024-09-11T07:29:55.327990+0000","last_unstale":"2024-09-11T07:30:00.013196+0000","last_undegraded":"2024-09-11T07:30:00.013196+0000","last_fullsized":"2024-09-11T07:30:00.013196+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:58.793358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"60'12","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185573+0000","last_change":"2024-09-11T07:29:51.229850+0000","last_active":"2024-09-11T07:29:59.185573+0000","last_peered":"2024-09-11T07:29:59.185573+0000","last_clean":"2024-09-11T07:29:59.185573+0000","last_became_active":"2024-09-11T07:29:51.229476+0000","last_became_peered":"2024-09-11T07:29:51.229476+0000","last_unstale":"2024-09-11T07:29:59.185573+0000","last_undegraded":"2024-09-11T07:29:59.185573+0000","last_fullsized":"2024-09-11T07:29:59.185573+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:47.473614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"53'2","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.186295+0000","last_change":"2024-09-11T07:29:49.219072+0000","last_active":"2024-09-11T07:29:59.186295+0000","last_peered":"2024-09-11T07:29:59.186295+0000","last_clean":"2024-09-11T07:29:59.186295+0000","last_became_active":"2024-09-11T07:29:49.218799+0000","last_became_peered":"2024-09-11T07:29:49.218799+0000","last_unstale":"2024-09-11T07:29:59.186295+0000","last_undegraded":"2024-09-11T07:29:59.186295+0000","last_fullsized":"2024-09-11T07:29:59.186295+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:19.018239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.600531+0000","last_change":"2024-09-11T07:29:53.249399+0000","last_active":"2024-09-11T07:29:59.600531+0000","last_peered":"2024-09-11T07:29:59.600531+0000","last_clean":"2024-09-11T07:29:59.600531+0000","last_became_active":"2024-09-11T07:29:53.249078+0000","last_became_peered":"2024-09-11T07:29:53.249078+0000","last_unstale":"2024-09-11T07:29:59.600531+0000","last_undegraded":"2024-09-11T07:29:59.600531+0000","last_fullsized":"2024-09-11T07:29:59.600531+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:39:03.017643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310756+0000","last_change":"2024-09-11T07:29:55.263490+0000","last_active":"2024-09-11T07:29:59.310756+0000","last_peered":"2024-09-11T07:29:59.310756+0000","last_clean":"2024-09-11T07:29:59.310756+0000","last_became_active":"2024-09-11T07:29:55.263152+0000","last_became_peered":"2024-09-11T07:29:55.263152+0000","last_unstale":"2024-09-11T07:29:59.310756+0000","last_undegraded":"2024-09-11T07:29:59.310756+0000","last_fullsized":"2024-09-11T07:29:59.310756+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:46.947000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192520+0000","last_change":"2024-09-11T07:29:51.250412+0000","last_active":"2024-09-11T07:29:59.192520+0000","last_peered":"2024-09-11T07:29:59.192520+0000","last_clean":"2024-09-11T07:29:59.192520+0000","last_became_active":"2024-09-11T07:29:51.250164+0000","last_became_peered":"2024-09-11T07:29:51.250164+0000","last_unstale":"2024-09-11T07:29:59.192520+0000","last_undegraded":"2024-09-11T07:29:59.192520+0000","last_fullsized":"2024-09-11T07:29:59.192520+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:18:16.491134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.600598+0000","last_change":"2024-09-11T07:29:49.214901+0000","last_active":"2024-09-11T07:29:59.600598+0000","last_peered":"2024-09-11T07:29:59.600598+0000","last_clean":"2024-09-11T07:29:59.600598+0000","last_became_active":"2024-09-11T07:29:49.214515+0000","last_became_peered":"2024-09-11T07:29:49.214515+0000","last_unstale":"2024-09-11T07:29:59.600598+0000","last_undegraded":"2024-09-11T07:29:59.600598+0000","last_fullsized":"2024-09-11T07:29:59.600598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:29:39.702003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.311099+0000","last_change":"2024-09-11T07:29:53.252723+0000","last_active":"2024-09-11T07:29:59.311099+0000","last_peered":"2024-09-11T07:29:59.311099+0000","last_clean":"2024-09-11T07:29:59.311099+0000","last_became_active":"2024-09-11T07:29:53.252437+0000","last_became_peered":"2024-09-11T07:29:53.252437+0000","last_unstale":"2024-09-11T07:29:59.311099+0000","last_undegraded":"2024-09-11T07:29:59.311099+0000","last_fullsized":"2024-09-11T07:29:59.311099+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:27.639699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.600956+0000","last_change":"2024-09-11T07:29:55.260575+0000","last_active":"2024-09-11T07:29:59.600956+0000","last_peered":"2024-09-11T07:29:59.600956+0000","last_clean":"2024-09-11T07:29:59.600956+0000","last_became_active":"2024-09-11T07:29:55.258857+0000","last_became_peered":"2024-09-11T07:29:55.258857+0000","last_unstale":"2024-09-11T07:29:59.600956+0000","last_undegraded":"2024-09-11T07:29:59.600956+0000","last_fullsized":"2024-09-11T07:29:59.600956+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:44:39.461312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.574494+0000","last_change":"2024-09-11T07:29:49.220644+0000","last_active":"2024-09-11T07:29:59.574494+0000","last_peered":"2024-09-11T07:29:59.574494+0000","last_clean":"2024-09-11T07:29:59.574494+0000","last_became_active":"2024-09-11T07:29:49.220396+0000","last_became_peered":"2024-09-11T07:29:49.220396+0000","last_unstale":"2024-09-11T07:29:59.574494+0000","last_undegraded":"2024-09-11T07:29:59.574494+0000","last_fullsized":"2024-09-11T07:29:59.574494+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:09:17.154821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"60'6","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.309705+0000","last_change":"2024-09-11T07:29:51.249491+0000","last_active":"2024-09-11T07:29:59.309705+0000","last_peered":"2024-09-11T07:29:59.309705+0000","last_clean":"2024-09-11T07:29:59.309705+0000","last_became_active":"2024-09-11T07:29:51.249250+0000","last_became_peered":"2024-09-11T07:29:51.249250+0000","last_unstale":"2024-09-11T07:29:59.309705+0000","last_undegraded":"2024-09-11T07:29:59.309705+0000","last_fullsized":"2024-09-11T07:29:59.309705+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:05:05.171133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013599+0000","last_change":"2024-09-11T07:29:53.247752+0000","last_active":"2024-09-11T07:30:00.013599+0000","last_peered":"2024-09-11T07:30:00.013599+0000","last_clean":"2024-09-11T07:30:00.013599+0000","last_became_active":"2024-09-11T07:29:53.247326+0000","last_became_peered":"2024-09-11T07:29:53.247326+0000","last_unstale":"2024-09-11T07:30:00.013599+0000","last_undegraded":"2024-09-11T07:30:00.013599+0000","last_fullsized":"2024-09-11T07:30:00.013599+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:58:37.862577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427580+0000","last_change":"2024-09-11T07:29:55.330684+0000","last_active":"2024-09-11T07:29:59.427580+0000","last_peered":"2024-09-11T07:29:59.427580+0000","last_clean":"2024-09-11T07:29:59.427580+0000","last_became_active":"2024-09-11T07:29:55.330269+0000","last_became_peered":"2024-09-11T07:29:55.330269+0000","last_unstale":"2024-09-11T07:29:59.427580+0000","last_undegraded":"2024-09-11T07:29:59.427580+0000","last_fullsized":"2024-09-11T07:29:59.427580+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:00.256204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.309933+0000","last_change":"2024-09-11T07:29:51.249466+0000","last_active":"2024-09-11T07:29:59.309933+0000","last_peered":"2024-09-11T07:29:59.309933+0000","last_clean":"2024-09-11T07:29:59.309933+0000","last_became_active":"2024-09-11T07:29:51.249175+0000","last_became_peered":"2024-09-11T07:29:51.249175+0000","last_unstale":"2024-09-11T07:29:59.309933+0000","last_undegraded":"2024-09-11T07:29:59.309933+0000","last_fullsized":"2024-09-11T07:29:59.309933+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:54.092983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013360+0000","last_change":"2024-09-11T07:29:49.222921+0000","last_active":"2024-09-11T07:30:00.013360+0000","last_peered":"2024-09-11T07:30:00.013360+0000","last_clean":"2024-09-11T07:30:00.013360+0000","last_became_active":"2024-09-11T07:29:49.222527+0000","last_became_peered":"2024-09-11T07:29:49.222527+0000","last_unstale":"2024-09-11T07:30:00.013360+0000","last_undegraded":"2024-09-11T07:30:00.013360+0000","last_fullsized":"2024-09-11T07:30:00.013360+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:10.176383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457456+0000","last_change":"2024-09-11T07:29:53.249033+0000","last_active":"2024-09-11T07:31:02.457456+0000","last_peered":"2024-09-11T07:31:02.457456+0000","last_clean":"2024-09-11T07:31:02.457456+0000","last_became_active":"2024-09-11T07:29:53.248776+0000","last_became_peered":"2024-09-11T07:29:53.248776+0000","last_unstale":"2024-09-11T07:31:02.457456+0000","last_undegraded":"2024-09-11T07:31:02.457456+0000","last_fullsized":"2024-09-11T07:31:02.457456+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:42.468271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192235+0000","last_change":"2024-09-11T07:29:55.268702+0000","last_active":"2024-09-11T07:29:59.192235+0000","last_peered":"2024-09-11T07:29:59.192235+0000","last_clean":"2024-09-11T07:29:59.192235+0000","last_became_active":"2024-09-11T07:29:55.268442+0000","last_became_peered":"2024-09-11T07:29:55.268442+0000","last_unstale":"2024-09-11T07:29:59.192235+0000","last_undegraded":"2024-09-11T07:29:59.192235+0000","last_fullsized":"2024-09-11T07:29:59.192235+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:57:37.788190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013624+0000","last_change":"2024-09-11T07:29:51.250427+0000","last_active":"2024-09-11T07:30:00.013624+0000","last_peered":"2024-09-11T07:30:00.013624+0000","last_clean":"2024-09-11T07:30:00.013624+0000","last_became_active":"2024-09-11T07:29:51.250080+0000","last_became_peered":"2024-09-11T07:29:51.250080+0000","last_unstale":"2024-09-11T07:30:00.013624+0000","last_undegraded":"2024-09-11T07:30:00.013624+0000","last_fullsized":"2024-09-11T07:30:00.013624+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:28:37.695830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192566+0000","last_change":"2024-09-11T07:29:49.221230+0000","last_active":"2024-09-11T07:29:59.192566+0000","last_peered":"2024-09-11T07:29:59.192566+0000","last_clean":"2024-09-11T07:29:59.192566+0000","last_became_active":"2024-09-11T07:29:49.220997+0000","last_became_peered":"2024-09-11T07:29:49.220997+0000","last_unstale":"2024-09-11T07:29:59.192566+0000","last_undegraded":"2024-09-11T07:29:59.192566+0000","last_fullsized":"2024-09-11T07:29:59.192566+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:51:30.322272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457578+0000","last_change":"2024-09-11T07:29:53.252769+0000","last_active":"2024-09-11T07:31:02.457578+0000","last_peered":"2024-09-11T07:31:02.457578+0000","last_clean":"2024-09-11T07:31:02.457578+0000","last_became_active":"2024-09-11T07:29:53.252492+0000","last_became_peered":"2024-09-11T07:29:53.252492+0000","last_unstale":"2024-09-11T07:31:02.457578+0000","last_undegraded":"2024-09-11T07:31:02.457578+0000","last_fullsized":"2024-09-11T07:31:02.457578+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:49:43.079885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.186086+0000","last_change":"2024-09-11T07:29:55.258735+0000","last_active":"2024-09-11T07:29:59.186086+0000","last_peered":"2024-09-11T07:29:59.186086+0000","last_clean":"2024-09-11T07:29:59.186086+0000","last_became_active":"2024-09-11T07:29:55.258364+0000","last_became_peered":"2024-09-11T07:29:55.258364+0000","last_unstale":"2024-09-11T07:29:59.186086+0000","last_undegraded":"2024-09-11T07:29:59.186086+0000","last_fullsized":"2024-09-11T07:29:59.186086+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:41.945015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.190538+0000","last_change":"2024-09-11T07:29:51.227078+0000","last_active":"2024-09-11T07:29:59.190538+0000","last_peered":"2024-09-11T07:29:59.190538+0000","last_clean":"2024-09-11T07:29:59.190538+0000","last_became_active":"2024-09-11T07:29:51.226767+0000","last_became_peered":"2024-09-11T07:29:51.226767+0000","last_unstale":"2024-09-11T07:29:59.190538+0000","last_undegraded":"2024-09-11T07:29:59.190538+0000","last_fullsized":"2024-09-11T07:29:59.190538+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:43:08.574720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.574661+0000","last_change":"2024-09-11T07:29:49.215959+0000","last_active":"2024-09-11T07:29:59.574661+0000","last_peered":"2024-09-11T07:29:59.574661+0000","last_clean":"2024-09-11T07:29:59.574661+0000","last_became_active":"2024-09-11T07:29:49.215548+0000","last_became_peered":"2024-09-11T07:29:49.215548+0000","last_unstale":"2024-09-11T07:29:59.574661+0000","last_undegraded":"2024-09-11T07:29:59.574661+0000","last_fullsized":"2024-09-11T07:29:59.574661+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:28:18.199146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310732+0000","last_change":"2024-09-11T07:29:53.253598+0000","last_active":"2024-09-11T07:29:59.310732+0000","last_peered":"2024-09-11T07:29:59.310732+0000","last_clean":"2024-09-11T07:29:59.310732+0000","last_became_active":"2024-09-11T07:29:53.253316+0000","last_became_peered":"2024-09-11T07:29:53.253316+0000","last_unstale":"2024-09-11T07:29:59.310732+0000","last_undegraded":"2024-09-11T07:29:59.310732+0000","last_fullsized":"2024-09-11T07:29:59.310732+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:15.550907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191010+0000","last_change":"2024-09-11T07:29:55.268457+0000","last_active":"2024-09-11T07:29:59.191010+0000","last_peered":"2024-09-11T07:29:59.191010+0000","last_clean":"2024-09-11T07:29:59.191010+0000","last_became_active":"2024-09-11T07:29:55.268043+0000","last_became_peered":"2024-09-11T07:29:55.268043+0000","last_unstale":"2024-09-11T07:29:59.191010+0000","last_undegraded":"2024-09-11T07:29:59.191010+0000","last_fullsized":"2024-09-11T07:29:59.191010+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:11:53.175252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185852+0000","last_change":"2024-09-11T07:29:51.227742+0000","last_active":"2024-09-11T07:29:59.185852+0000","last_peered":"2024-09-11T07:29:59.185852+0000","last_clean":"2024-09-11T07:29:59.185852+0000","last_became_active":"2024-09-11T07:29:51.227338+0000","last_became_peered":"2024-09-11T07:29:51.227338+0000","last_unstale":"2024-09-11T07:29:59.185852+0000","last_undegraded":"2024-09-11T07:29:59.185852+0000","last_fullsized":"2024-09-11T07:29:59.185852+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:58.190819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192121+0000","last_change":"2024-09-11T07:29:49.213948+0000","last_active":"2024-09-11T07:29:59.192121+0000","last_peered":"2024-09-11T07:29:59.192121+0000","last_clean":"2024-09-11T07:29:59.192121+0000","last_became_active":"2024-09-11T07:29:49.213688+0000","last_became_peered":"2024-09-11T07:29:49.213688+0000","last_unstale":"2024-09-11T07:29:59.192121+0000","last_undegraded":"2024-09-11T07:29:59.192121+0000","last_fullsized":"2024-09-11T07:29:59.192121+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:55:58.520871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.190407+0000","last_change":"2024-09-11T07:29:53.247092+0000","last_active":"2024-09-11T07:29:59.190407+0000","last_peered":"2024-09-11T07:29:59.190407+0000","last_clean":"2024-09-11T07:29:59.190407+0000","last_became_active":"2024-09-11T07:29:53.246734+0000","last_became_peered":"2024-09-11T07:29:53.246734+0000","last_unstale":"2024-09-11T07:29:59.190407+0000","last_undegraded":"2024-09-11T07:29:59.190407+0000","last_fullsized":"2024-09-11T07:29:59.190407+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:54.215825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310287+0000","last_change":"2024-09-11T07:29:55.265565+0000","last_active":"2024-09-11T07:29:59.310287+0000","last_peered":"2024-09-11T07:29:59.310287+0000","last_clean":"2024-09-11T07:29:59.310287+0000","last_became_active":"2024-09-11T07:29:55.265297+0000","last_became_peered":"2024-09-11T07:29:55.265297+0000","last_unstale":"2024-09-11T07:29:59.310287+0000","last_undegraded":"2024-09-11T07:29:59.310287+0000","last_fullsized":"2024-09-11T07:29:59.310287+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:08:24.061874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"60'4","reported_seq":22,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.309585+0000","last_change":"2024-09-11T07:29:51.228237+0000","last_active":"2024-09-11T07:29:59.309585+0000","last_peered":"2024-09-11T07:29:59.309585+0000","last_clean":"2024-09-11T07:29:59.309585+0000","last_became_active":"2024-09-11T07:29:51.227964+0000","last_became_peered":"2024-09-11T07:29:51.227964+0000","last_unstale":"2024-09-11T07:29:59.309585+0000","last_undegraded":"2024-09-11T07:29:59.309585+0000","last_fullsized":"2024-09-11T07:29:59.309585+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:09:44.080239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013694+0000","last_change":"2024-09-11T07:29:49.212211+0000","last_active":"2024-09-11T07:30:00.013694+0000","last_peered":"2024-09-11T07:30:00.013694+0000","last_clean":"2024-09-11T07:30:00.013694+0000","last_became_active":"2024-09-11T07:29:49.211946+0000","last_became_peered":"2024-09-11T07:29:49.211946+0000","last_unstale":"2024-09-11T07:30:00.013694+0000","last_undegraded":"2024-09-11T07:30:00.013694+0000","last_fullsized":"2024-09-11T07:30:00.013694+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:29:26.088357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.574391+0000","last_change":"2024-09-11T07:29:53.243979+0000","last_active":"2024-09-11T07:29:59.574391+0000","last_peered":"2024-09-11T07:29:59.574391+0000","last_clean":"2024-09-11T07:29:59.574391+0000","last_became_active":"2024-09-11T07:29:53.243683+0000","last_became_peered":"2024-09-11T07:29:53.243683+0000","last_unstale":"2024-09-11T07:29:59.574391+0000","last_undegraded":"2024-09-11T07:29:59.574391+0000","last_fullsized":"2024-09-11T07:29:59.574391+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:35:47.027425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427414+0000","last_change":"2024-09-11T07:29:55.258631+0000","last_active":"2024-09-11T07:29:59.427414+0000","last_peered":"2024-09-11T07:29:59.427414+0000","last_clean":"2024-09-11T07:29:59.427414+0000","last_became_active":"2024-09-11T07:29:55.258336+0000","last_became_peered":"2024-09-11T07:29:55.258336+0000","last_unstale":"2024-09-11T07:29:59.427414+0000","last_undegraded":"2024-09-11T07:29:59.427414+0000","last_fullsized":"2024-09-11T07:29:59.427414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:33.039223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310402+0000","last_change":"2024-09-11T07:29:51.250044+0000","last_active":"2024-09-11T07:29:59.310402+0000","last_peered":"2024-09-11T07:29:59.310402+0000","last_clean":"2024-09-11T07:29:59.310402+0000","last_became_active":"2024-09-11T07:29:51.249096+0000","last_became_peered":"2024-09-11T07:29:51.249096+0000","last_unstale":"2024-09-11T07:29:59.310402+0000","last_undegraded":"2024-09-11T07:29:59.310402+0000","last_fullsized":"2024-09-11T07:29:59.310402+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:57.444345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.575095+0000","last_change":"2024-09-11T07:29:49.216411+0000","last_active":"2024-09-11T07:29:59.575095+0000","last_peered":"2024-09-11T07:29:59.575095+0000","last_clean":"2024-09-11T07:29:59.575095+0000","last_became_active":"2024-09-11T07:29:49.216066+0000","last_became_peered":"2024-09-11T07:29:49.216066+0000","last_unstale":"2024-09-11T07:29:59.575095+0000","last_undegraded":"2024-09-11T07:29:59.575095+0000","last_fullsized":"2024-09-11T07:29:59.575095+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:24.592043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427312+0000","last_change":"2024-09-11T07:29:53.244412+0000","last_active":"2024-09-11T07:29:59.427312+0000","last_peered":"2024-09-11T07:29:59.427312+0000","last_clean":"2024-09-11T07:29:59.427312+0000","last_became_active":"2024-09-11T07:29:53.244161+0000","last_became_peered":"2024-09-11T07:29:53.244161+0000","last_unstale":"2024-09-11T07:29:59.427312+0000","last_undegraded":"2024-09-11T07:29:59.427312+0000","last_fullsized":"2024-09-11T07:29:59.427312+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:54:21.328805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310556+0000","last_change":"2024-09-11T07:29:55.329242+0000","last_active":"2024-09-11T07:29:59.310556+0000","last_peered":"2024-09-11T07:29:59.310556+0000","last_clean":"2024-09-11T07:29:59.310556+0000","last_became_active":"2024-09-11T07:29:55.328900+0000","last_became_peered":"2024-09-11T07:29:55.328900+0000","last_unstale":"2024-09-11T07:29:59.310556+0000","last_undegraded":"2024-09-11T07:29:59.310556+0000","last_fullsized":"2024-09-11T07:29:59.310556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:03.159527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"60'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.427747+0000","last_change":"2024-09-11T07:29:55.261644+0000","last_active":"2024-09-11T07:29:59.427747+0000","last_peered":"2024-09-11T07:29:59.427747+0000","last_clean":"2024-09-11T07:29:59.427747+0000","last_became_active":"2024-09-11T07:29:55.261382+0000","last_became_peered":"2024-09-11T07:29:55.261382+0000","last_unstale":"2024-09-11T07:29:59.427747+0000","last_undegraded":"2024-09-11T07:29:59.427747+0000","last_fullsized":"2024-09-11T07:29:59.427747+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:40.086338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"53'1","reported_seq":27,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:30:00.013146+0000","last_change":"2024-09-11T07:29:49.221452+0000","last_active":"2024-09-11T07:30:00.013146+0000","last_peered":"2024-09-11T07:30:00.013146+0000","last_clean":"2024-09-11T07:30:00.013146+0000","last_became_active":"2024-09-11T07:29:49.221186+0000","last_became_peered":"2024-09-11T07:29:49.221186+0000","last_unstale":"2024-09-11T07:30:00.013146+0000","last_undegraded":"2024-09-11T07:30:00.013146+0000","last_fullsized":"2024-09-11T07:30:00.013146+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:51:39.004180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192520+0000","last_change":"2024-09-11T07:29:51.251244+0000","last_active":"2024-09-11T07:29:59.192520+0000","last_peered":"2024-09-11T07:29:59.192520+0000","last_clean":"2024-09-11T07:29:59.192520+0000","last_became_active":"2024-09-11T07:29:51.251000+0000","last_became_peered":"2024-09-11T07:29:51.251000+0000","last_unstale":"2024-09-11T07:29:59.192520+0000","last_undegraded":"2024-09-11T07:29:59.192520+0000","last_fullsized":"2024-09-11T07:29:59.192520+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:32:08.729452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"60'8","reported_seq":41,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T07:31:02.457457+0000","last_change":"2024-09-11T07:29:53.244528+0000","last_active":"2024-09-11T07:31:02.457457+0000","last_peered":"2024-09-11T07:31:02.457457+0000","last_clean":"2024-09-11T07:31:02.457457+0000","last_became_active":"2024-09-11T07:29:53.244235+0000","last_became_peered":"2024-09-11T07:29:53.244235+0000","last_unstale":"2024-09-11T07:31:02.457457+0000","last_undegraded":"2024-09-11T07:31:02.457457+0000","last_fullsized":"2024-09-11T07:31:02.457457+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:04:52.385829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191597+0000","last_change":"2024-09-11T07:29:55.263902+0000","last_active":"2024-09-11T07:29:59.191597+0000","last_peered":"2024-09-11T07:29:59.191597+0000","last_clean":"2024-09-11T07:29:59.191597+0000","last_became_active":"2024-09-11T07:29:55.263630+0000","last_became_peered":"2024-09-11T07:29:55.263630+0000","last_unstale":"2024-09-11T07:29:59.191597+0000","last_undegraded":"2024-09-11T07:29:59.191597+0000","last_fullsized":"2024-09-11T07:29:59.191597+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:26.902446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310537+0000","last_change":"2024-09-11T07:29:49.218872+0000","last_active":"2024-09-11T07:29:59.310537+0000","last_peered":"2024-09-11T07:29:59.310537+0000","last_clean":"2024-09-11T07:29:59.310537+0000","last_became_active":"2024-09-11T07:29:49.218553+0000","last_became_peered":"2024-09-11T07:29:49.218553+0000","last_unstale":"2024-09-11T07:29:59.310537+0000","last_undegraded":"2024-09-11T07:29:59.310537+0000","last_fullsized":"2024-09-11T07:29:59.310537+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:04:29.145363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.574275+0000","last_change":"2024-09-11T07:29:51.232028+0000","last_active":"2024-09-11T07:29:59.574275+0000","last_peered":"2024-09-11T07:29:59.574275+0000","last_clean":"2024-09-11T07:29:59.574275+0000","last_became_active":"2024-09-11T07:29:51.231721+0000","last_became_peered":"2024-09-11T07:29:51.231721+0000","last_unstale":"2024-09-11T07:29:59.574275+0000","last_undegraded":"2024-09-11T07:29:59.574275+0000","last_fullsized":"2024-09-11T07:29:59.574275+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:19.153130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.192365+0000","last_change":"2024-09-11T07:29:53.251201+0000","last_active":"2024-09-11T07:29:59.192365+0000","last_peered":"2024-09-11T07:29:59.192365+0000","last_clean":"2024-09-11T07:29:59.192365+0000","last_became_active":"2024-09-11T07:29:53.250915+0000","last_became_peered":"2024-09-11T07:29:53.250915+0000","last_unstale":"2024-09-11T07:29:59.192365+0000","last_undegraded":"2024-09-11T07:29:59.192365+0000","last_fullsized":"2024-09-11T07:29:59.192365+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:56:21.448567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.186103+0000","last_change":"2024-09-11T07:29:55.328222+0000","last_active":"2024-09-11T07:29:59.186103+0000","last_peered":"2024-09-11T07:29:59.186103+0000","last_clean":"2024-09-11T07:29:59.186103+0000","last_became_active":"2024-09-11T07:29:55.327928+0000","last_became_peered":"2024-09-11T07:29:55.327928+0000","last_unstale":"2024-09-11T07:29:59.186103+0000","last_undegraded":"2024-09-11T07:29:59.186103+0000","last_fullsized":"2024-09-11T07:29:59.186103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:51:22.629378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.574543+0000","last_change":"2024-09-11T07:29:49.224002+0000","last_active":"2024-09-11T07:29:59.574543+0000","last_peered":"2024-09-11T07:29:59.574543+0000","last_clean":"2024-09-11T07:29:59.574543+0000","last_became_active":"2024-09-11T07:29:49.223718+0000","last_became_peered":"2024-09-11T07:29:49.223718+0000","last_unstale":"2024-09-11T07:29:59.574543+0000","last_undegraded":"2024-09-11T07:29:59.574543+0000","last_fullsized":"2024-09-11T07:29:59.574543+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:32.072683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.600246+0000","last_change":"2024-09-11T07:29:51.229685+0000","last_active":"2024-09-11T07:29:59.600246+0000","last_peered":"2024-09-11T07:29:59.600246+0000","last_clean":"2024-09-11T07:29:59.600246+0000","last_became_active":"2024-09-11T07:29:51.229440+0000","last_became_peered":"2024-09-11T07:29:51.229440+0000","last_unstale":"2024-09-11T07:29:59.600246+0000","last_undegraded":"2024-09-11T07:29:59.600246+0000","last_fullsized":"2024-09-11T07:29:59.600246+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:27:33.077762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.191892+0000","last_change":"2024-09-11T07:29:53.248964+0000","last_active":"2024-09-11T07:29:59.191892+0000","last_peered":"2024-09-11T07:29:59.191892+0000","last_clean":"2024-09-11T07:29:59.191892+0000","last_became_active":"2024-09-11T07:29:53.246107+0000","last_became_peered":"2024-09-11T07:29:53.246107+0000","last_unstale":"2024-09-11T07:29:59.191892+0000","last_undegraded":"2024-09-11T07:29:59.191892+0000","last_fullsized":"2024-09-11T07:29:59.191892+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:17.721563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.311050+0000","last_change":"2024-09-11T07:29:55.328603+0000","last_active":"2024-09-11T07:29:59.311050+0000","last_peered":"2024-09-11T07:29:59.311050+0000","last_clean":"2024-09-11T07:29:59.311050+0000","last_became_active":"2024-09-11T07:29:55.328174+0000","last_became_peered":"2024-09-11T07:29:55.328174+0000","last_unstale":"2024-09-11T07:29:59.311050+0000","last_undegraded":"2024-09-11T07:29:59.311050+0000","last_fullsized":"2024-09-11T07:29:59.311050+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:54.236689+0000","last_clean_scrub_stamp":"2024-09-11T07:29:54.236689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:20:21.186656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.310883+0000","last_change":"2024-09-11T07:29:49.219357+0000","last_active":"2024-09-11T07:29:59.310883+0000","last_peered":"2024-09-11T07:29:59.310883+0000","last_clean":"2024-09-11T07:29:59.310883+0000","last_became_active":"2024-09-11T07:29:49.218848+0000","last_became_peered":"2024-09-11T07:29:49.218848+0000","last_unstale":"2024-09-11T07:29:59.310883+0000","last_undegraded":"2024-09-11T07:29:59.310883+0000","last_fullsized":"2024-09-11T07:29:59.310883+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.196835+0000","last_clean_scrub_stamp":"2024-09-11T07:29:48.196835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:22:14.344430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"60'12","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.309644+0000","last_change":"2024-09-11T07:29:51.227143+0000","last_active":"2024-09-11T07:29:59.309644+0000","last_peered":"2024-09-11T07:29:59.309644+0000","last_clean":"2024-09-11T07:29:59.309644+0000","last_became_active":"2024-09-11T07:29:51.226709+0000","last_became_peered":"2024-09-11T07:29:51.226709+0000","last_unstale":"2024-09-11T07:29:59.309644+0000","last_undegraded":"2024-09-11T07:29:59.309644+0000","last_fullsized":"2024-09-11T07:29:59.309644+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:50.207783+0000","last_clean_scrub_stamp":"2024-09-11T07:29:50.207783+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:50:59.006747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:29:59.185910+0000","last_change":"2024-09-11T07:29:53.248591+0000","last_active":"2024-09-11T07:29:59.185910+0000","last_peered":"2024-09-11T07:29:59.185910+0000","last_clean":"2024-09-11T07:29:59.185910+0000","last_became_active":"2024-09-11T07:29:53.248319+0000","last_became_peered":"2024-09-11T07:29:53.248319+0000","last_unstale":"2024-09-11T07:29:59.185910+0000","last_undegraded":"2024-09-11T07:29:59.185910+0000","last_fullsized":"2024-09-11T07:29:59.185910+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:52.223879+0000","last_clean_scrub_stamp":"2024-09-11T07:29:52.223879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:27.733005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397525,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298544,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446608,"statfs":{"total":95995035648,"available":95689326592,"internally_reserved":0,"allocated":1220608,"data_stored":826179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.33700000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.36699999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.502}]},{"osd":4,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.52700000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.42399999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.439},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51500000000000001}]}]},{"osd":6,"up_from":43,"seq":184683593754,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298516,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446636,"statfs":{"total":95995035648,"available":95689355264,"internally_reserved":0,"allocated":1191936,"data_stored":824040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.436},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.44400000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.34599999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.46400000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.315}]},{"osd":5,"last update":"Wed Sep 11 07:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.36399999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:30:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.45200000000000001}]}]},{"osd":5,"up_from":37,"seq":158913789982,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297948,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447204,"statfs":{"total":95995035648,"available":95689936896,"internally_reserved":0,"allocated":606208,"data_stored":234542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.497,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.61899999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.38300000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.47399999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.624,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51300000000000001,"15min":0.51900000000000002},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.59599999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.622,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.36899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70299999999999996,"5min":0.751,"15min":0.751},"last":0.45100000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.497,"15min":0.495},"min":{"1min":0.40200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67500000000000004,"5min":0.748,"15min":0.748},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.48799999999999999,"15min":0.48899999999999999},"min":{"1min":0.374,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.52800000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.375,"15min":0.36899999999999999},"min":{"1min":0.26700000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.39300000000000002},"min":{"1min":0.28599999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63100000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.29099999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:30:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.27400000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.311},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.374}]}]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297968,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447184,"statfs":{"total":95995035648,"available":95689916416,"internally_reserved":0,"allocated":626688,"data_stored":234998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:30:29 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.51700000000000002,"15min":0.52700000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.63,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.73399999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.26400000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:30:29 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.504,"15min":0.50600000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70599999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70999999999999996,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.38200000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:30:29 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52300000000000002,"15min":0.52700000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69199999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.312,"5min":0.309,"15min":0.309},"max":{"1min":0.74399999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.33500000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:30:29 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.74399999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.68000000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.28699999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:30:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.33500000000000002},"min":{"1min":0.26900000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.60199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.34999999999999998,"15min":0.34000000000000002},"min":{"1min":0.31,"5min":0.255,"15min":0.255},"max":{"1min":0.55700000000000005,"5min":0.62,"15min":0.62},"last":0.51500000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:30:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.47599999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:30:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.502},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.433}]}]},{"osd":3,"up_from":26,"seq":111669149736,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297984,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447168,"statfs":{"total":95995035648,"available":95689900032,"internally_reserved":0,"allocated":643072,"data_stored":234790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:30:08 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.622,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.247}]},{"osd":1,"last update":"Wed Sep 11 07:30:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.40600000000000003,"15min":0.40699999999999997},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.51600000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.216},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.26600000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.52100000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.26500000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:30:08 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.438,"15min":0.442},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.54900000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40600000000000003,"15min":0.40699999999999997},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:30:33 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.42499999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:30:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35599999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:30:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.33400000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:30:40 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.373},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39300000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297948,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447204,"statfs":{"total":95995035648,"available":95689936896,"internally_reserved":0,"allocated":606208,"data_stored":234778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:30:53 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.37,"15min":0.38},"min":{"1min":0.216,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46999999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.314},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.39100000000000001,"15min":0.39900000000000002},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.53700000000000003,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.28399999999999997}]},{"osd":1,"last update":"Wed Sep 11 07:30:53 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.40000000000000002,"15min":0.40899999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60599999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.247},{"interface":"front","average":{"1min":0.378,"5min":0.40899999999999997,"15min":0.41899999999999998},"min":{"1min":0.26800000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.55700000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.41699999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:30:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.41799999999999998,"15min":0.42199999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.443,"15min":0.44900000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.59499999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.36799999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:30:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68300000000000005,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.40300000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:30:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53000000000000003,"15min":0.52800000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.37,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46000000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:30:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.54200000000000004}]},{"osd":7,"last update":"Wed Sep 11 07:30:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.55700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297996,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447156,"statfs":{"total":95995035648,"available":95689887744,"internally_reserved":0,"allocated":655360,"data_stored":235587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:30:29 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.33300000000000002,"15min":0.317},"min":{"1min":0.222,"5min":0.184,"15min":0.184},"max":{"1min":0.55600000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.20200000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.33800000000000002,"15min":0.32000000000000001},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.65100000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.26100000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:30:55 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.371,"15min":0.36499999999999999},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.52200000000000002,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.371,"15min":0.36499999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.53000000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.222}]},{"osd":3,"last update":"Wed Sep 11 07:30:11 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.53400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.248},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.438,"15min":0.441},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64000000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.32400000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:30:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.746,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.72199999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.65400000000000003}]},{"osd":5,"last update":"Wed Sep 11 07:30:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.53300000000000003,"15min":0.54000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72899999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.35199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84099999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.49099999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:30:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.628},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57199999999999995}]},{"osd":7,"last update":"Wed Sep 11 07:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.55000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298536,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446616,"statfs":{"total":95995035648,"available":95689334784,"internally_reserved":0,"allocated":1212416,"data_stored":824640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34100000000000003,"15min":0.33000000000000002},"min":{"1min":0.27600000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.46600000000000003,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.375},{"interface":"front","average":{"1min":0.379,"5min":0.35799999999999998,"15min":0.34399999999999997},"min":{"1min":0.26500000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.42999999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.38600000000000001,"15min":0.38700000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.46400000000000002,"5min":0.62,"15min":0.62},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.38200000000000001,"15min":0.378},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48599999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.50800000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:30:13 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.32500000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41099999999999998,"15min":0.41299999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.48599999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69699999999999995,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55600000000000005,"15min":0.56100000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.65500000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.53400000000000003}]},{"osd":5,"last update":"Wed Sep 11 07:30:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.65900000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.747,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.56299999999999994}]},{"osd":6,"last update":"Wed Sep 11 07:30:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58499999999999996}]},{"osd":7,"last update":"Wed Sep 11 07:30:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.626},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.59899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:31:16.727 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T07:31:16.727 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T07:31:16.728 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T07:31:16.728 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T07:31:16.746 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=plugin.loader t=2024-09-11T07:31:16.484064629Z level=info msg="Plugin registered" pluginID=input 2024-09-11T07:31:16.746 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=plugin.loader t=2024-09-11T07:31:16.51108669Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-11T07:31:16.746 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=plugin.loader t=2024-09-11T07:31:16.511108788Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-11T07:31:16.746 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=secrets t=2024-09-11T07:31:16.511143331Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-11T07:31:16.746 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=query_data t=2024-09-11T07:31:16.521725044Z level=info msg="Query Service initialization" 2024-09-11T07:31:16.746 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=live.push_http t=2024-09-11T07:31:16.523948486Z level=info msg="Live Push Gateway initialization" 2024-09-11T07:31:17.005 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:17.017 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:16 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:17.017 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:16 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:17.018 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:16 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:17.018 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:16 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:17.018 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:16 smithi143 ceph-mon[29604]: Deploying daemon node-exporter.a on smithi143 2024-09-11T07:31:17.018 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:16 smithi143 ceph-mon[29604]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:31:17.018 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:16 smithi143 ceph-mon[29604]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:17.018 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:16 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:17.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:16 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:17.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:16 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:17.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:16 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:17.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:16 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:17.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:16 smithi143 ceph-mon[38451]: Deploying daemon node-exporter.a on smithi143 2024-09-11T07:31:17.019 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:16 smithi143 ceph-mon[38451]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:31:17.019 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:16 smithi143 ceph-mon[38451]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:17.019 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:16 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:17.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:17.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:17.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:17.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:17.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-mon[33167]: Deploying daemon node-exporter.a on smithi143 2024-09-11T07:31:17.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-mon[33167]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:31:17.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-mon[33167]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:17.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:16 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:17.717 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=infra.usagestats.collector t=2024-09-11T07:31:17.280121699Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-11T07:31:17.717 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=provisioning.datasources t=2024-09-11T07:31:17.280405317Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-11T07:31:17.717 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=provisioning.datasources t=2024-09-11T07:31:17.312785658Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-11T07:31:17.717 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=provisioning.datasources t=2024-09-11T07:31:17.344211259Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-11T07:31:17.717 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=provisioning.alerting t=2024-09-11T07:31:17.365793314Z level=info msg="starting to provision alerting" 2024-09-11T07:31:17.717 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=provisioning.alerting t=2024-09-11T07:31:17.365819034Z level=info msg="finished to provision alerting" 2024-09-11T07:31:17.717 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=grafanaStorageLogger t=2024-09-11T07:31:17.365996995Z level=info msg="storage starting" 2024-09-11T07:31:17.717 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=ngalert.state.manager t=2024-09-11T07:31:17.366020335Z level=info msg="Warming state cache for startup" 2024-09-11T07:31:17.717 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=http.server t=2024-09-11T07:31:17.367771819Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-11T07:31:17.718 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=sqlstore.transactions t=2024-09-11T07:31:17.376578144Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T07:31:17.718 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=ngalert.state.manager t=2024-09-11T07:31:17.37665774Z level=info msg="State cache has been initialized" states=0 duration=10.636085ms 2024-09-11T07:31:17.718 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=ngalert.multiorg.alertmanager t=2024-09-11T07:31:17.376718458Z level=info msg="starting MultiOrg Alertmanager" 2024-09-11T07:31:17.718 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=ticker t=2024-09-11T07:31:17.376738792Z level=info msg=starting first_tick=2024-09-11T07:31:20Z 2024-09-11T07:31:17.718 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=sqlstore.transactions t=2024-09-11T07:31:17.43252925Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T07:31:17.718 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=sqlstore.transactions t=2024-09-11T07:31:17.443813126Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-11T07:31:17.718 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=sqlstore.transactions t=2024-09-11T07:31:17.45512611Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T07:31:17.718 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:31:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a[66785]: logger=sqlstore.transactions t=2024-09-11T07:31:17.481067947Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T07:31:18.815 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-11T07:31:18.816 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T07:31:19.250 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:19 smithi143 ceph-mon[29604]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:19.250 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:19 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3413715771' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T07:31:19.250 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:19 smithi143 ceph-mon[38451]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:19.250 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:19 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3413715771' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T07:31:19.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:19 smithi164 ceph-mon[33167]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:19.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:19 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3413715771' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T07:31:19.871 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T07:31:19.871 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T07:31:19.871 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T07:31:19.883 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi143.front.sepia.ceph.com 2024-09-11T07:31:19.884 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d3dfd926-700e-11ef-bcea-c7b262605968 -- bash -c 'set -ex 2024-09-11T07:31:19.884 DEBUG:teuthology.orchestra.run.smithi143:> for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T07:31:19.884 DEBUG:teuthology.orchestra.run.smithi143:> do 2024-09-11T07:31:19.884 DEBUG:teuthology.orchestra.run.smithi143:> echo "rotating key for $f" 2024-09-11T07:31:19.884 DEBUG:teuthology.orchestra.run.smithi143:> K=$(ceph auth get-key $f) 2024-09-11T07:31:19.884 DEBUG:teuthology.orchestra.run.smithi143:> NK="$K" 2024-09-11T07:31:19.884 DEBUG:teuthology.orchestra.run.smithi143:> ceph orch daemon rotate-key $f 2024-09-11T07:31:19.884 DEBUG:teuthology.orchestra.run.smithi143:> while [ "$K" == "$NK" ]; do 2024-09-11T07:31:19.885 DEBUG:teuthology.orchestra.run.smithi143:> sleep 5 2024-09-11T07:31:19.885 DEBUG:teuthology.orchestra.run.smithi143:> NK=$(ceph auth get-key $f) 2024-09-11T07:31:19.885 DEBUG:teuthology.orchestra.run.smithi143:> done 2024-09-11T07:31:19.885 DEBUG:teuthology.orchestra.run.smithi143:> done 2024-09-11T07:31:19.885 DEBUG:teuthology.orchestra.run.smithi143:> ' 2024-09-11T07:31:20.649 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:21.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:21 smithi143 ceph-mon[29604]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:21.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:21 smithi143 ceph-mon[38451]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:21.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:21 smithi164 ceph-mon[33167]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:21.467 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:31:21 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:31:21.799 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:21 smithi143 podman[74461]: 2024-09-11 07:31:21.557376893 +0000 UTC m=+3.801474675 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T07:31:22.069 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:21 smithi143 podman[74461]: 2024-09-11 07:31:21.814066681 +0000 UTC m=+4.058164500 container create a75708e36fc0873f1d7d039364f2fc4d8ab310a9ce33c7642f9068fa589fcaf6 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:22.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:22 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:22.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:22.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:22.549 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 podman[74461]: 2024-09-11 07:31:22.091304865 +0000 UTC m=+4.335402642 container init a75708e36fc0873f1d7d039364f2fc4d8ab310a9ce33c7642f9068fa589fcaf6 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:22.549 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 podman[74461]: 2024-09-11 07:31:22.096627775 +0000 UTC m=+4.340725555 container start a75708e36fc0873f1d7d039364f2fc4d8ab310a9ce33c7642f9068fa589fcaf6 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:22.549 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.101Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T07:31:22.549 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.101Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T07:31:22.549 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.101Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T07:31:22.550 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.101Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T07:31:22.550 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T07:31:22.550 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T07:31:22.550 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T07:31:22.550 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T07:31:22.550 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T07:31:22.550 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T07:31:22.550 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T07:31:22.550 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T07:31:22.550 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T07:31:22.550 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T07:31:22.550 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T07:31:22.550 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T07:31:22.550 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T07:31:22.551 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T07:31:22.551 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T07:31:22.551 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T07:31:22.551 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T07:31:22.551 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T07:31:22.551 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T07:31:22.551 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T07:31:22.551 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T07:31:22.551 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T07:31:22.551 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T07:31:22.551 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T07:31:22.551 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T07:31:22.551 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T07:31:22.552 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T07:31:22.552 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T07:31:22.552 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T07:31:22.552 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T07:31:22.552 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T07:31:22.552 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T07:31:22.552 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T07:31:22.552 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T07:31:22.552 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T07:31:22.552 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T07:31:22.552 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T07:31:22.553 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T07:31:22.553 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T07:31:22.553 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T07:31:22.553 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T07:31:22.553 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T07:31:22.553 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T07:31:22.553 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T07:31:22.553 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T07:31:22.553 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T07:31:22.553 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T07:31:22.553 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T07:31:22.553 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a[74733]: ts=2024-09-11T07:31:22.102Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T07:31:22.553 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 bash[74461]: a75708e36fc0873f1d7d039364f2fc4d8ab310a9ce33c7642f9068fa589fcaf6 2024-09-11T07:31:22.554 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:31:22 smithi143 systemd[1]: Started Ceph node-exporter.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:31:23.131 INFO:teuthology.orchestra.run.smithi143.stdout:rotating key for osd.0 2024-09-11T07:31:23.131 INFO:teuthology.orchestra.run.smithi143.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T07:31:23.131 INFO:teuthology.orchestra.run.smithi143.stderr:+ echo 'rotating key for osd.0' 2024-09-11T07:31:23.131 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:31:23.216 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:23 smithi164 bash[67063]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-11T07:31:23.358 INFO:teuthology.orchestra.run.smithi143.stderr:+ K=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:31:23.358 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:31:23.358 INFO:teuthology.orchestra.run.smithi143.stderr:+ ceph orch daemon rotate-key osd.0 2024-09-11T07:31:23.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:23 smithi143 ceph-mon[38451]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:23.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:23 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:23.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:23 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:23.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:23 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:23.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:23 smithi143 ceph-mon[38451]: Deploying daemon node-exporter.b on smithi164 2024-09-11T07:31:23.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:23 smithi143 ceph-mon[29604]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:23.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:23 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:23.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:23 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:23.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:23 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:23.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:23 smithi143 ceph-mon[29604]: Deploying daemon node-exporter.b on smithi164 2024-09-11T07:31:23.612 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled to rotate-key osd.0 on host 'smithi143' 2024-09-11T07:31:23.623 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:31:23.623 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:31:23.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:23 smithi164 ceph-mon[33167]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:23.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:23 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:23.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:23 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:23.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:23 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:23.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:23 smithi164 ceph-mon[33167]: Deploying daemon node-exporter.b on smithi164 2024-09-11T07:31:23.717 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:23 smithi164 bash[67063]: Getting image source signatures 2024-09-11T07:31:23.717 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:23 smithi164 bash[67063]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-11T07:31:23.717 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:23 smithi164 bash[67063]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T07:31:23.717 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:23 smithi164 bash[67063]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-11T07:31:24.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:24 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1073480813' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:24.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:24 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:24.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:24 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:24.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:24 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1073480813' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:24.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:24 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:24.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:24 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:24.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:24 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1073480813' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:24.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:24 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:24.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:24 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:24.717 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:31:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:31:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:31:25.172 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:24 smithi164 bash[67063]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T07:31:25.172 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:24 smithi164 bash[67063]: Writing manifest to image destination 2024-09-11T07:31:25.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-mon[33167]: from='client.24596 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:31:25.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-mon[33167]: Schedule rotate-key daemon osd.0 2024-09-11T07:31:25.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-mon[33167]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:25.467 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 podman[67063]: 2024-09-11 07:31:25.147679262 +0000 UTC m=+2.100350743 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T07:31:25.467 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 podman[67063]: 2024-09-11 07:31:25.268506733 +0000 UTC m=+2.221178260 container create b4feb6ba9bf94d8dc8696e5aefbb06605d5b5cb0671b110e9674fb159728d69a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:31:25.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:25 smithi143 ceph-mon[29604]: from='client.24596 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:31:25.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:25 smithi143 ceph-mon[29604]: Schedule rotate-key daemon osd.0 2024-09-11T07:31:25.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:25 smithi143 ceph-mon[29604]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:25.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:25 smithi143 ceph-mon[38451]: from='client.24596 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:31:25.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:25 smithi143 ceph-mon[38451]: Schedule rotate-key daemon osd.0 2024-09-11T07:31:25.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:25 smithi143 ceph-mon[38451]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:25.967 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 podman[67063]: 2024-09-11 07:31:25.54601554 +0000 UTC m=+2.498687025 container init b4feb6ba9bf94d8dc8696e5aefbb06605d5b5cb0671b110e9674fb159728d69a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:31:25.967 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 podman[67063]: 2024-09-11 07:31:25.551352572 +0000 UTC m=+2.504024055 container start b4feb6ba9bf94d8dc8696e5aefbb06605d5b5cb0671b110e9674fb159728d69a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:31:25.967 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.555Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T07:31:25.967 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.555Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T07:31:25.967 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.555Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T07:31:25.967 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T07:31:25.967 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T07:31:25.967 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T07:31:25.967 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T07:31:25.967 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T07:31:25.968 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T07:31:25.968 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T07:31:25.968 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T07:31:25.968 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T07:31:25.968 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T07:31:25.968 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T07:31:25.968 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T07:31:25.968 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T07:31:25.969 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T07:31:25.969 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T07:31:25.969 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T07:31:25.969 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T07:31:25.969 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T07:31:25.969 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T07:31:25.969 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T07:31:25.969 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T07:31:25.969 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T07:31:25.969 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T07:31:25.969 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T07:31:25.969 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T07:31:25.969 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T07:31:25.970 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T07:31:25.970 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T07:31:25.970 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T07:31:25.970 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T07:31:25.970 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T07:31:25.970 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T07:31:25.970 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T07:31:25.970 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T07:31:25.970 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T07:31:25.970 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T07:31:25.970 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T07:31:25.970 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T07:31:25.970 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T07:31:25.970 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T07:31:25.971 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T07:31:25.971 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T07:31:25.971 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T07:31:25.971 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T07:31:25.971 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T07:31:25.971 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T07:31:25.971 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T07:31:25.971 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T07:31:25.971 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T07:31:25.971 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b[67138]: ts=2024-09-11T07:31:25.556Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T07:31:25.971 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 bash[67063]: b4feb6ba9bf94d8dc8696e5aefbb06605d5b5cb0671b110e9674fb159728d69a 2024-09-11T07:31:25.971 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:31:25 smithi164 systemd[1]: Started Ceph node-exporter.b for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:31:26.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:31:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2024-09-11T07:31:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-11T07:31:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-11T07:31:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-11T07:31:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:26 smithi164 ceph-mon[33167]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:26.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:27.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:27.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:27.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:27.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:27.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:27.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:31:27.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2024-09-11T07:31:27.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-11T07:31:27.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-11T07:31:27.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-11T07:31:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[29604]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:27.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:27.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:27.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:27.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:27.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:27.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:31:27.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2024-09-11T07:31:27.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-11T07:31:27.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-11T07:31:27.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-11T07:31:27.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[38451]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:27.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:27.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:28.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:27 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]': finished 2024-09-11T07:31:28.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:27 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-11T07:31:28.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:27 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-09-11T07:31:28.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:27 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-11T07:31:28.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:27 smithi143 ceph-mon[29604]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T07:31:28.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:27 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]': finished 2024-09-11T07:31:28.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:27 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-11T07:31:28.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:27 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-09-11T07:31:28.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:27 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-11T07:31:28.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:27 smithi143 ceph-mon[38451]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T07:31:28.072 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:27 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]': finished 2024-09-11T07:31:28.073 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:27 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-11T07:31:28.073 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:27 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-09-11T07:31:28.073 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:27 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-11T07:31:28.073 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:27 smithi164 ceph-mon[33167]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T07:31:28.625 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:31:28.935 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:31:28.935 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:31:28.935 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:31:28.935 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:28 smithi143 ceph-mon[29604]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T07:31:28.936 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:28 smithi143 ceph-mon[29604]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:31:28.936 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:28 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:28.936 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:28 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:28.936 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:28 smithi143 ceph-mon[38451]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T07:31:28.936 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:28 smithi143 ceph-mon[38451]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:31:28.936 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:28 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:28.936 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:28 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:29.103 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:28 smithi164 ceph-mon[33167]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T07:31:29.103 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:28 smithi164 ceph-mon[33167]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:31:29.103 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:28 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:29.103 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:28 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:30.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:29 smithi143 ceph-mon[29604]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-11T07:31:30.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:29 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3872755432' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:30.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:29 smithi143 ceph-mon[38451]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-11T07:31:30.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:29 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3872755432' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:30.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:29 smithi164 ceph-mon[33167]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-11T07:31:30.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:29 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3872755432' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:31.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:31 smithi143 ceph-mon[29604]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:31:31.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:31 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:31.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:31 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:31.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:31 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:31:31.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:31 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:31:31.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:31 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:31.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:31 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-11T07:31:31.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:31 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-11T07:31:31.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:31 smithi143 ceph-mon[38451]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:31:31.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:31 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:31.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:31 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:31.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:31 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:31:31.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:31 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:31:31.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:31 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:31.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:31 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-11T07:31:31.301 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:31 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-11T07:31:31.466 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:31:31 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:31:31.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:31 smithi164 ceph-mon[33167]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:31:31.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:31 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:31.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:31 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:31.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:31 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:31:31.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:31 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:31:31.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:31 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:31.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:31 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-11T07:31:31.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:31 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-11T07:31:32.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:32 smithi164 ceph-mon[33167]: Rotating authentication key for osd.0 2024-09-11T07:31:32.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:32 smithi164 ceph-mon[33167]: Reconfiguring daemon osd.0 on smithi143 2024-09-11T07:31:32.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:32 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:32.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:32 smithi143 ceph-mon[29604]: Rotating authentication key for osd.0 2024-09-11T07:31:32.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:32 smithi143 ceph-mon[29604]: Reconfiguring daemon osd.0 on smithi143 2024-09-11T07:31:32.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:32 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:32.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:32 smithi143 ceph-mon[38451]: Rotating authentication key for osd.0 2024-09-11T07:31:32.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:32 smithi143 ceph-mon[38451]: Reconfiguring daemon osd.0 on smithi143 2024-09-11T07:31:32.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:32 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:33.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:33 smithi164 ceph-mon[33167]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 136 B/s, 1 objects/s recovering 2024-09-11T07:31:33.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:33 smithi164 ceph-mon[33167]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-11T07:31:33.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:33 smithi164 ceph-mon[33167]: Cluster is now healthy 2024-09-11T07:31:33.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:33 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:33.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:33 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:33.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:33 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:33.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:33 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:33.489 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:33 smithi143 ceph-mon[29604]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 136 B/s, 1 objects/s recovering 2024-09-11T07:31:33.489 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:33 smithi143 ceph-mon[29604]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-11T07:31:33.489 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:33 smithi143 ceph-mon[29604]: Cluster is now healthy 2024-09-11T07:31:33.489 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:33 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:33.489 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:33 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:33.489 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:33 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:33.489 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:33 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:33.490 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:33 smithi143 ceph-mon[38451]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 136 B/s, 1 objects/s recovering 2024-09-11T07:31:33.490 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:33 smithi143 ceph-mon[38451]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-11T07:31:33.490 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:33 smithi143 ceph-mon[38451]: Cluster is now healthy 2024-09-11T07:31:33.490 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:33 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:33.490 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:33 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:33.490 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:33 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:33.490 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:33 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:33.937 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:31:34.246 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:31:34.246 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:31:34.247 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:31:34.461 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:34 smithi143 ceph-mon[29604]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T07:31:34.461 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:34 smithi143 ceph-mon[29604]: Reconfiguring daemon alertmanager.a on smithi143 2024-09-11T07:31:34.461 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:34 smithi143 ceph-mon[38451]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T07:31:34.461 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:34 smithi143 ceph-mon[38451]: Reconfiguring daemon alertmanager.a on smithi143 2024-09-11T07:31:34.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:34 smithi164 ceph-mon[33167]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T07:31:34.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:34 smithi164 ceph-mon[33167]: Reconfiguring daemon alertmanager.a on smithi143 2024-09-11T07:31:34.467 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:31:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:31:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:31:35.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:35 smithi164 ceph-mon[33167]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 136 B/s, 1 objects/s recovering 2024-09-11T07:31:35.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:35 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/604813080' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:35.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:35 smithi143 ceph-mon[29604]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 136 B/s, 1 objects/s recovering 2024-09-11T07:31:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:35 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/604813080' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:35.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:35 smithi143 ceph-mon[38451]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 136 B/s, 1 objects/s recovering 2024-09-11T07:31:35.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:35 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/604813080' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:36.015 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:35 smithi143 systemd[1]: Stopping Ceph alertmanager.a for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:31:36.298 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:36 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[73059]: ts=2024-09-11T07:31:36.013Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T07:31:36.298 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:36 smithi143 podman[75905]: 2024-09-11 07:31:36.034395719 +0000 UTC m=+0.116148987 container died 2ef0e3968f2476da7241bd64e63acbe8abefcf13677beb65258e07a60f19054d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:36.755 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:36 smithi143 podman[75905]: 2024-09-11 07:31:36.511498365 +0000 UTC m=+0.593251674 container cleanup 2ef0e3968f2476da7241bd64e63acbe8abefcf13677beb65258e07a60f19054d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:36.755 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:36 smithi143 bash[75905]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T07:31:37.048 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:36 smithi143 podman[75922]: 2024-09-11 07:31:36.754046118 +0000 UTC m=+0.721522821 container remove 2ef0e3968f2476da7241bd64e63acbe8abefcf13677beb65258e07a60f19054d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:37.049 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:36 smithi143 podman[75922]: 2024-09-11 07:31:36.838207478 +0000 UTC m=+0.805684168 volume remove b95e708d5a1671a8d82dfa69a373c318645635f4fe0844faba5b3c6566870bfd 2024-09-11T07:31:37.049 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:36 smithi143 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T07:31:37.049 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:36 smithi143 systemd[1]: Stopped Ceph alertmanager.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:31:37.049 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:36 smithi143 systemd[1]: Starting Ceph alertmanager.a for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:31:37.332 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:37 smithi143 ceph-mon[29604]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 117 B/s, 0 objects/s recovering 2024-09-11T07:31:37.332 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:37 smithi143 ceph-mon[38451]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 117 B/s, 0 objects/s recovering 2024-09-11T07:31:37.333 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:37 smithi143 podman[76023]: 2024-09-11 07:31:37.0911876 +0000 UTC m=+0.016106410 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-11T07:31:37.333 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:37 smithi143 podman[76023]: 2024-09-11 07:31:37.222583236 +0000 UTC m=+0.147502082 volume create 6574e6fc7bb75d7f1742a279cdd1141ab53feb07b3443e6c00dd3591a30b4412 2024-09-11T07:31:37.333 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:37 smithi143 podman[76023]: 2024-09-11 07:31:37.331411759 +0000 UTC m=+0.256330603 container create 6d2fef2e434be6ab36c0ac3c441e70dfd4ff1a3fe9c8dfba2f374577d8a6a919 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:37.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:37 smithi164 ceph-mon[33167]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 117 B/s, 0 objects/s recovering 2024-09-11T07:31:38.052 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:37 smithi143 podman[76023]: 2024-09-11 07:31:37.63399891 +0000 UTC m=+0.558917724 container init 6d2fef2e434be6ab36c0ac3c441e70dfd4ff1a3fe9c8dfba2f374577d8a6a919 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:38.053 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:37 smithi143 podman[76023]: 2024-09-11 07:31:37.638686811 +0000 UTC m=+0.563605612 container start 6d2fef2e434be6ab36c0ac3c441e70dfd4ff1a3fe9c8dfba2f374577d8a6a919 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:38.053 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:37 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[76038]: ts=2024-09-11T07:31:37.649Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-11T07:31:38.053 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:37 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[76038]: ts=2024-09-11T07:31:37.649Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-11T07:31:38.053 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:37 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[76038]: ts=2024-09-11T07:31:37.651Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.143 port=9094 2024-09-11T07:31:38.053 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:37 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[76038]: ts=2024-09-11T07:31:37.652Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-11T07:31:38.053 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:37 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[76038]: ts=2024-09-11T07:31:37.676Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T07:31:38.053 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:37 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[76038]: ts=2024-09-11T07:31:37.676Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T07:31:38.053 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:37 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[76038]: ts=2024-09-11T07:31:37.678Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-11T07:31:38.053 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:37 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[76038]: ts=2024-09-11T07:31:37.678Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-11T07:31:38.053 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:37 smithi143 bash[76023]: 6d2fef2e434be6ab36c0ac3c441e70dfd4ff1a3fe9c8dfba2f374577d8a6a919 2024-09-11T07:31:38.054 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:37 smithi143 systemd[1]: Started Ceph alertmanager.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:31:39.025 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:38 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:39.025 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:38 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:39.025 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:38 smithi143 ceph-mon[29604]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T07:31:39.025 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:38 smithi143 ceph-mon[29604]: Reconfiguring daemon prometheus.a on smithi164 2024-09-11T07:31:39.025 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:38 smithi143 ceph-mon[29604]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 99 B/s wr, 3 op/s; 106 B/s, 0 objects/s recovering 2024-09-11T07:31:39.025 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:38 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:39.026 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:38 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:39.026 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:38 smithi143 ceph-mon[38451]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T07:31:39.026 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:38 smithi143 ceph-mon[38451]: Reconfiguring daemon prometheus.a on smithi164 2024-09-11T07:31:39.026 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:38 smithi143 ceph-mon[38451]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 99 B/s wr, 3 op/s; 106 B/s, 0 objects/s recovering 2024-09-11T07:31:39.026 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:38 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:39.026 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:38 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:39.026 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:38 smithi164 ceph-mon[33167]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T07:31:39.026 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:38 smithi164 ceph-mon[33167]: Reconfiguring daemon prometheus.a on smithi164 2024-09-11T07:31:39.026 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:38 smithi164 ceph-mon[33167]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 99 B/s wr, 3 op/s; 106 B/s, 0 objects/s recovering 2024-09-11T07:31:39.249 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:31:39.562 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:31:39.562 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:31:39.562 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:31:40.002 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:39 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1608789384' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:40.002 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:39 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[76038]: ts=2024-09-11T07:31:39.652Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000379641s 2024-09-11T07:31:40.002 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:39 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1608789384' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:40.038 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:39 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1608789384' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:41.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:40 smithi143 ceph-mon[29604]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s; 91 B/s, 0 objects/s recovering 2024-09-11T07:31:41.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:40 smithi143 ceph-mon[38451]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s; 91 B/s, 0 objects/s recovering 2024-09-11T07:31:41.190 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:40 smithi164 ceph-mon[33167]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s; 91 B/s, 0 objects/s recovering 2024-09-11T07:31:41.191 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:40 smithi164 systemd[1]: Stopping Ceph prometheus.a for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:31:41.467 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:31:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:31:41.467 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:31:41.188Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T07:31:41.467 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:31:41.188Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T07:31:41.467 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:31:41.188Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T07:31:41.467 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:31:41.188Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T07:31:41.467 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:31:41.189Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T07:31:41.467 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:31:41.189Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T07:31:41.467 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:31:41.189Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T07:31:41.467 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:31:41.189Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T07:31:41.467 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:31:41.189Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T07:31:41.467 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:31:41.189Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T07:31:41.468 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:31:41.189Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T07:31:41.468 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[64847]: ts=2024-09-11T07:31:41.189Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T07:31:41.468 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:41 smithi164 podman[68168]: 2024-09-11 07:31:41.199287169 +0000 UTC m=+0.130556825 container died 55c09ced2ef69c97dd9ead9f397957cca3a365e6f8f301603a4c81d7ae8a9a1d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:41.778 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:41 smithi164 podman[68168]: 2024-09-11 07:31:41.60363241 +0000 UTC m=+0.534902060 container cleanup 55c09ced2ef69c97dd9ead9f397957cca3a365e6f8f301603a4c81d7ae8a9a1d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:41.778 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:41 smithi164 bash[68168]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T07:31:42.029 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:41 smithi164 podman[68188]: 2024-09-11 07:31:41.779910774 +0000 UTC m=+0.581968181 container remove 55c09ced2ef69c97dd9ead9f397957cca3a365e6f8f301603a4c81d7ae8a9a1d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:42.030 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:41 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:42.030 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:41 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:42.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:41 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:42.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:41 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:42.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:41 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:42.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:41 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:42.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-11T07:31:42.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 systemd[1]: Stopped Ceph prometheus.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:31:42.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.107s CPU time. 2024-09-11T07:31:42.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 systemd[1]: Starting Ceph prometheus.a for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:31:42.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 podman[68284]: 2024-09-11 07:31:42.14651336 +0000 UTC m=+0.021203418 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-11T07:31:42.362 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 podman[68284]: 2024-09-11 07:31:42.247726478 +0000 UTC m=+0.122416582 container create 966edf34c783a662dc5aaac963f9063ec79f720b10e0f94b6128b33d788c85e1 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:42.788 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 podman[68284]: 2024-09-11 07:31:42.49976146 +0000 UTC m=+0.374451520 container init 966edf34c783a662dc5aaac963f9063ec79f720b10e0f94b6128b33d788c85e1 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:42.788 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 podman[68284]: 2024-09-11 07:31:42.504539199 +0000 UTC m=+0.379229258 container start 966edf34c783a662dc5aaac963f9063ec79f720b10e0f94b6128b33d788c85e1 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:31:42.788 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.529Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T07:31:42.789 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.529Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T07:31:42.789 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.529Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi164 (none))" 2024-09-11T07:31:42.789 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.529Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T07:31:42.789 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.529Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T07:31:42.789 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.530Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T07:31:42.789 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.530Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T07:31:42.789 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.531Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T07:31:42.789 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.531Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T07:31:42.789 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.533Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T07:31:42.789 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.534Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.102µs 2024-09-11T07:31:42.789 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.534Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T07:31:42.789 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.534Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-11T07:31:42.789 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.534Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-11T07:31:42.790 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.534Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=20.171µs wal_replay_duration=471.415µs wbl_replay_duration=97ns total_replay_duration=510.477µs 2024-09-11T07:31:42.790 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.535Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T07:31:42.790 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.535Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T07:31:42.790 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.535Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T07:31:42.790 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.550Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.921799ms db_storage=753ns remote_storage=1.08µs web_handler=411ns query_engine=594ns scrape=394.768µs scrape_sd=28.22µs notify=23.533µs notify_sd=8.793µs rules=14.231818ms tracing=5.626µs 2024-09-11T07:31:42.790 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.550Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T07:31:42.790 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:31:42.550Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T07:31:42.790 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 bash[68284]: 966edf34c783a662dc5aaac963f9063ec79f720b10e0f94b6128b33d788c85e1 2024-09-11T07:31:42.790 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:31:42 smithi164 systemd[1]: Started Ceph prometheus.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:31:42.879 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:42] ENGINE Bus STOPPING 2024-09-11T07:31:42.879 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:42] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:31:42.879 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[29604]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 3 op/s; 91 B/s, 0 objects/s recovering 2024-09-11T07:31:42.879 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:42.879 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:42.879 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:31:42.879 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi143.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:31:42.879 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:42.880 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:31:42.880 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi164.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:31:42.880 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:42.880 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:31:42.880 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi164.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:31:42.880 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:42.880 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:31:42.880 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[38451]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 3 op/s; 91 B/s, 0 objects/s recovering 2024-09-11T07:31:42.880 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:42.880 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:42.880 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:31:42.881 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi143.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:31:42.881 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:42.881 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:31:42.881 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi164.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:31:42.881 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:42.881 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:31:42.881 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi164.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:31:42.881 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:42.881 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:31:43.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-mon[33167]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 3 op/s; 91 B/s, 0 objects/s recovering 2024-09-11T07:31:43.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:43.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:43.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:31:43.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi143.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:31:43.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:43.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:31:43.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi164.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:31:43.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:43.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:31:43.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi164.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:31:43.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:43.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:42 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:31:43.295 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:42] ENGINE Bus STOPPED 2024-09-11T07:31:43.295 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:42] ENGINE Bus STARTING 2024-09-11T07:31:43.295 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:42] ENGINE Serving on http://:::9283 2024-09-11T07:31:43.296 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:42] ENGINE Bus STARTED 2024-09-11T07:31:43.296 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:42 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:42] ENGINE Bus STOPPING 2024-09-11T07:31:43.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:43] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:31:43.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:43] ENGINE Bus STOPPED 2024-09-11T07:31:43.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:43] ENGINE Bus STARTING 2024-09-11T07:31:44.048 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:43] ENGINE Serving on http://:::9283 2024-09-11T07:31:44.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:43] ENGINE Bus STARTED 2024-09-11T07:31:44.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:43] ENGINE Bus STOPPING 2024-09-11T07:31:44.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-mon[29604]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:31:44.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-mon[29604]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi143.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:31:44.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-mon[29604]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:31:44.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-mon[29604]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi164.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:31:44.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-mon[29604]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:31:44.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-mon[29604]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi164.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:31:44.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-mon[38451]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:31:44.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-mon[38451]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi143.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:31:44.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-mon[38451]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:31:44.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-mon[38451]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi164.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:31:44.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-mon[38451]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:31:44.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:43 smithi143 ceph-mon[38451]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi164.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:31:44.182 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:43 smithi164 ceph-mon[33167]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:31:44.182 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:43 smithi164 ceph-mon[33167]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi143.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:31:44.182 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:43 smithi164 ceph-mon[33167]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:31:44.182 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:43 smithi164 ceph-mon[33167]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi164.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:31:44.182 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:43 smithi164 ceph-mon[33167]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:31:44.182 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:43 smithi164 ceph-mon[33167]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi164.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:31:44.382 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:44 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:44] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:31:44.382 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:44 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:44] ENGINE Bus STOPPED 2024-09-11T07:31:44.382 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:44 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:44] ENGINE Bus STARTING 2024-09-11T07:31:44.383 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:44 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:44] ENGINE Serving on http://:::9283 2024-09-11T07:31:44.383 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:44 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:31:44] ENGINE Bus STARTED 2024-09-11T07:31:44.564 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:31:44.876 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:31:44.904 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:31:44.904 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:31:45.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:44 smithi143 ceph-mon[29604]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:45.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:44 smithi143 ceph-mon[38451]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:45.095 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:44 smithi164 ceph-mon[33167]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:46.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:45 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3238227219' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:46.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:45 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:46.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:45 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:46.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:45 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3238227219' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:46.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:45 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:46.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:45 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:46.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:45 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3238227219' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:46.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:45 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:46.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:45 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:47.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:46 smithi143 ceph-mon[29604]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:47.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:46 smithi143 ceph-mon[38451]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:47.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:46 smithi164 ceph-mon[33167]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:48.048 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:31:47 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[76038]: ts=2024-09-11T07:31:47.656Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003560404s 2024-09-11T07:31:49.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:49 smithi164 ceph-mon[33167]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:49 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:49 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:49 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:31:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:49 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:31:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:49 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:49 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:31:49.490 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:49 smithi143 ceph-mon[38451]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:49.490 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:49 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:49.490 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:49 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:49.490 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:49 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:31:49.490 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:49 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:31:49.490 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:49 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:49.490 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:49 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:31:49.490 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:49 smithi143 ceph-mon[29604]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:49.490 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:49 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:49.490 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:49 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:49.490 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:49 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:31:49.490 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:49 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:31:49.491 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:49 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:49.491 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:49 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:31:49.877 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:31:50.243 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:31:50.244 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:31:50.244 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:31:51.413 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:31:51 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:31:51.414 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:51 smithi164 ceph-mon[33167]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:51.414 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:51 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1493338755' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:51.438 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:51 smithi143 ceph-mon[29604]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:51.438 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:51 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1493338755' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:51.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:51 smithi143 ceph-mon[38451]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:51.438 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:51 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1493338755' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:52.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:52 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:52.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:52 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:52.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:52 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:52.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:52 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:52.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:52 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:52.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:52 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:52.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:52 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:52.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:52 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:52.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:52 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:53.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:53 smithi164 ceph-mon[33167]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:53.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:53 smithi143 ceph-mon[29604]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:53.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:53 smithi143 ceph-mon[38451]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:54.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:31:54 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:31:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:31:54.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:54 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:54.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:54 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:54.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:54 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:31:54.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:54 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:31:54.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:54 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:54.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:54 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:54.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:54 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:54.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:54 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:31:54.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:54 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:31:54.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:54 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:54.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:54 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:54.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:54 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:54.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:54 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:31:54.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:54 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:31:54.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:54 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:31:55.245 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:31:55.410 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:31:55 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:31:55] "GET /metrics HTTP/1.1" 200 219823 "" "Prometheus/2.43.0" 2024-09-11T07:31:55.557 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:31:55.557 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:31:55.557 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:31:55.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:55 smithi164 ceph-mon[33167]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:55.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:55 smithi143 ceph-mon[29604]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:55.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:55 smithi143 ceph-mon[38451]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:56.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:56 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2052754074' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:56.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:56 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:56.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:56 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2052754074' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:56.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:56 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:56.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:56 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2052754074' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:31:56.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:56 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:57.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:57 smithi164 ceph-mon[33167]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:57.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:57 smithi143 ceph-mon[29604]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:57.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:57 smithi143 ceph-mon[38451]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:31:59.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:31:59 smithi164 ceph-mon[33167]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:59.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:31:59 smithi143 ceph-mon[29604]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:31:59.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:31:59 smithi143 ceph-mon[38451]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:00.559 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:32:00.872 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:32:00.872 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:32:00.872 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:32:01.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:01 smithi164 ceph-mon[33167]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:01.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:01 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/34122350' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:01.467 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:32:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:32:01.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:01 smithi143 ceph-mon[29604]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:01.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:01 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/34122350' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:01.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:01 smithi143 ceph-mon[38451]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:01.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:01 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/34122350' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:02.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:02 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:02.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:02 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:02.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:02 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:03.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:03 smithi164 ceph-mon[33167]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:03.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:03 smithi143 ceph-mon[29604]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:03.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:03 smithi143 ceph-mon[38451]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:04.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:32:04 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:32:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:32:05.433 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:32:05 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:32:05] "GET /metrics HTTP/1.1" 200 219799 "" "Prometheus/2.43.0" 2024-09-11T07:32:05.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:05 smithi164 ceph-mon[33167]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:05.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:05 smithi143 ceph-mon[29604]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:05.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:05 smithi143 ceph-mon[38451]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:05.874 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:32:06.188 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:32:06.188 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:32:06.188 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:32:06.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:06 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1717768562' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:06.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:06 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1717768562' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:06.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:06 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1717768562' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:07.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:07 smithi164 ceph-mon[33167]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:07.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:07 smithi143 ceph-mon[29604]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:07.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:07 smithi143 ceph-mon[38451]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:09.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:09 smithi164 ceph-mon[33167]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:09.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:09 smithi143 ceph-mon[29604]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:09.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:09 smithi143 ceph-mon[38451]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:11.191 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:32:11.505 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:32:11.505 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:32:11.506 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:32:11.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:11 smithi164 ceph-mon[33167]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:11.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:11 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:11.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:32:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:32:11.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:11 smithi143 ceph-mon[38451]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:11.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:11 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:11.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:11 smithi143 ceph-mon[29604]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:11.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:11 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:12.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:12 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:12.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:12 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2503663070' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:12.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:12 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:12.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:12 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2503663070' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:12.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:12 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:12.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:12 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2503663070' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:13.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:13 smithi164 ceph-mon[33167]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:13.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:13 smithi143 ceph-mon[29604]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:13.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:13 smithi143 ceph-mon[38451]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:14.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:32:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:32:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:32:15.461 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:32:15 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:32:15] "GET /metrics HTTP/1.1" 200 219770 "" "Prometheus/2.43.0" 2024-09-11T07:32:15.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:15 smithi164 ceph-mon[33167]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:15.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:15 smithi143 ceph-mon[29604]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:15.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:15 smithi143 ceph-mon[38451]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:16.507 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:32:16.821 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:32:16.821 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:32:16.821 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:32:17.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:17 smithi164 ceph-mon[33167]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:17.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:17 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/505640805' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:17.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:17 smithi143 ceph-mon[38451]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:17.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:17 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/505640805' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:17.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:17 smithi143 ceph-mon[29604]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:17.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:17 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/505640805' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:19.749 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:19 smithi143 ceph-mon[29604]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:19.750 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:19 smithi143 ceph-mon[38451]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:19.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:19 smithi164 ceph-mon[33167]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:21.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:21 smithi164 ceph-mon[33167]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:21.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:32:21 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:32:21.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:21 smithi143 ceph-mon[29604]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:21.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:21 smithi143 ceph-mon[38451]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:21.823 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:32:22.137 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:32:22.137 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:32:22.137 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:32:22.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:22 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:22.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:22 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3081025327' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:22.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:22 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:22.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:22 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3081025327' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:22.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:22 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:22.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:22 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3081025327' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:23.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:23 smithi143 ceph-mon[29604]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:23.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:23 smithi143 ceph-mon[38451]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:23.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:23 smithi164 ceph-mon[33167]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:24.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:32:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:32:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:32:25.482 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:32:25 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:32:25] "GET /metrics HTTP/1.1" 200 219770 "" "Prometheus/2.43.0" 2024-09-11T07:32:25.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:25 smithi143 ceph-mon[29604]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:25.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:25 smithi143 ceph-mon[38451]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:25.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:25 smithi164 ceph-mon[33167]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:26.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:26.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:26.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:27.139 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:32:27.462 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:32:27.462 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:32:27.462 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:32:27.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:27 smithi143 ceph-mon[38451]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:27.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:27 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2928269388' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:27.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:27 smithi143 ceph-mon[29604]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:27.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:27 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2928269388' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:27.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:27 smithi164 ceph-mon[33167]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:27.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:27 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2928269388' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:29.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:29 smithi143 ceph-mon[29604]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:29.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:29 smithi143 ceph-mon[38451]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:29.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:29 smithi164 ceph-mon[33167]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:31.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:31 smithi164 ceph-mon[33167]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:31.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:32:31 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:32:31.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:31 smithi143 ceph-mon[29604]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:31.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:31 smithi143 ceph-mon[38451]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:32.464 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:32:32.780 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:32:32.780 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:32:32.780 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:32:32.781 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:32 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:32.781 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:32 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:32.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:32 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:33.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:33 smithi143 ceph-mon[29604]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:33.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:33 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2357680967' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:33.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:33 smithi143 ceph-mon[38451]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:33.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:33 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2357680967' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:33.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:33 smithi164 ceph-mon[33167]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:33.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:33 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2357680967' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:34.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:32:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:32:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:32:35.506 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:32:35 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:32:35] "GET /metrics HTTP/1.1" 200 219766 "" "Prometheus/2.43.0" 2024-09-11T07:32:35.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:35 smithi143 ceph-mon[29604]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:35.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:35 smithi143 ceph-mon[38451]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:35.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:35 smithi164 ceph-mon[33167]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:37.782 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:32:37.782 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:37 smithi143 ceph-mon[29604]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:37.783 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:37 smithi143 ceph-mon[38451]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:37.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:37 smithi164 ceph-mon[33167]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:38.097 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:32:38.097 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:32:38.097 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:32:38.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:38 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1261076727' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:38.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:38 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1261076727' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:38.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:38 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1261076727' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:39.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:39 smithi143 ceph-mon[29604]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:39.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:39 smithi143 ceph-mon[38451]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:39.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:39 smithi164 ceph-mon[33167]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:41.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:41 smithi164 ceph-mon[33167]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:41.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:41 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:41.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:32:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:32:41.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:41 smithi143 ceph-mon[29604]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:41.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:41 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:41.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:41 smithi143 ceph-mon[38451]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:41.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:41 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:42.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:42 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:42.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:42 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:42.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:42 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:43.098 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:32:43.412 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:32:43.412 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:32:43.412 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:32:43.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:43 smithi143 ceph-mon[29604]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:43.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:43 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3730712117' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:43.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:43 smithi143 ceph-mon[38451]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:43.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:43 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3730712117' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:43.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:43 smithi164 ceph-mon[33167]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:43.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:43 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3730712117' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:44.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:32:44 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:32:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:32:45.533 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:32:45 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:32:45] "GET /metrics HTTP/1.1" 200 219778 "" "Prometheus/2.43.0" 2024-09-11T07:32:45.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:45 smithi143 ceph-mon[29604]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:45.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:45 smithi143 ceph-mon[38451]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:45.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:45 smithi164 ceph-mon[33167]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:46.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:46 smithi164 ceph-mon[33167]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:47.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:46 smithi143 ceph-mon[29604]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:47.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:46 smithi143 ceph-mon[38451]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:48.417 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:32:48.727 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:32:48.727 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:32:48.727 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:32:49.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:49 smithi164 ceph-mon[33167]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:49.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:49 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3358500613' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:49.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:49 smithi143 ceph-mon[29604]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:49.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:49 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3358500613' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:49.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:49 smithi143 ceph-mon[38451]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:49.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:49 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3358500613' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:51.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:51 smithi164 ceph-mon[33167]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:51.467 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:32:51 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:32:51.548 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:51 smithi143 ceph-mon[38451]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:51.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:51 smithi143 ceph-mon[29604]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:52.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:52 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:52.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:52 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:52.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:52 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:53.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:53 smithi164 ceph-mon[33167]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:53.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:53 smithi143 ceph-mon[29604]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:53.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:53 smithi143 ceph-mon[38451]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:53.730 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:32:54.060 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:32:54.061 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:32:54.061 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:32:54.396 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:54 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:32:54.397 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:54 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:32:54.397 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:54 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:32:54.397 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:54 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:32:54.397 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:54 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2316908861' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:54.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:54 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:32:54.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:54 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:32:54.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:54 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:32:54.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:54 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:32:54.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:54 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2316908861' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:54.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:54 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:32:54.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:54 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:32:54.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:54 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:32:54.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:54 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:32:54.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:54 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2316908861' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:32:54.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:32:54 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:32:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:32:55.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:55 smithi164 ceph-mon[33167]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:55.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:55 smithi143 ceph-mon[29604]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:55.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:32:55 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:32:55] "GET /metrics HTTP/1.1" 200 219778 "" "Prometheus/2.43.0" 2024-09-11T07:32:55.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:55 smithi143 ceph-mon[38451]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:56.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:56 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:56.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:56 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:56.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:56 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:57.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:57 smithi164 ceph-mon[33167]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:57.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:57 smithi143 ceph-mon[29604]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:57.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:57 smithi143 ceph-mon[38451]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:32:59.061 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:32:59.375 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:32:59.376 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:32:59.376 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:32:59.376 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:32:59 smithi143 ceph-mon[29604]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:59.376 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:32:59 smithi143 ceph-mon[38451]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:32:59.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:32:59 smithi164 ceph-mon[33167]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:00.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:00 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/827191619' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:00.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:00 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/827191619' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:00.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:00 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/827191619' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:01.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:01 smithi164 ceph-mon[33167]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:01.467 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:33:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:33:01.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:01 smithi143 ceph-mon[29604]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:01.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:01 smithi143 ceph-mon[38451]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:02.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:02 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:02.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:02 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:02.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:02 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:03.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:03 smithi164 ceph-mon[33167]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:03.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:03 smithi143 ceph-mon[29604]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:03.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:03 smithi143 ceph-mon[38451]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:04.377 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:33:04.688 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:33:04.688 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:33:04.689 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:33:04.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:33:04 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:33:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:33:05.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:05 smithi164 ceph-mon[33167]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:05.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:05 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3514624748' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:05.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:05 smithi143 ceph-mon[29604]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:05.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:05 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3514624748' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:05.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:33:05 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:33:05] "GET /metrics HTTP/1.1" 200 219782 "" "Prometheus/2.43.0" 2024-09-11T07:33:05.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:05 smithi143 ceph-mon[38451]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:05.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:05 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3514624748' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:07.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:07 smithi164 ceph-mon[33167]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:07.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:07 smithi143 ceph-mon[29604]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:07.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:07 smithi143 ceph-mon[38451]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:09.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:09 smithi164 ceph-mon[33167]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:09.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:09 smithi143 ceph-mon[29604]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:09.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:09 smithi143 ceph-mon[38451]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:09.691 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:33:10.001 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:33:10.002 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:33:10.002 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:33:10.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:10 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1751868338' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:10.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:10 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1751868338' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:10.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:10 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1751868338' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:11.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:11 smithi164 ceph-mon[33167]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:11.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:11 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:11.467 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:33:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:33:11.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:11 smithi143 ceph-mon[38451]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:11.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:11 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:11.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:11 smithi143 ceph-mon[29604]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:11.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:11 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:12.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:12 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:12.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:12 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:12.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:12 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:13.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:13 smithi164 ceph-mon[33167]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:13.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:13 smithi143 ceph-mon[29604]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:13.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:13 smithi143 ceph-mon[38451]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:14.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:33:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:33:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:33:15.003 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:33:15.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:15 smithi143 ceph-mon[29604]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:15.299 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:33:15 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:33:15] "GET /metrics HTTP/1.1" 200 219782 "" "Prometheus/2.43.0" 2024-09-11T07:33:15.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:15 smithi143 ceph-mon[38451]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:15.319 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:33:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:33:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:33:15.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:15 smithi164 ceph-mon[33167]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:16.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:16 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/4230693008' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:16.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:16 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/4230693008' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:16.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:16 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/4230693008' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:17.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:17 smithi164 ceph-mon[33167]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:17.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:17 smithi143 ceph-mon[29604]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:17.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:17 smithi143 ceph-mon[38451]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:19.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:19 smithi164 ceph-mon[33167]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:19.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:19 smithi143 ceph-mon[29604]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:19.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:19 smithi143 ceph-mon[38451]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:20.322 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:33:20.632 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:33:20.633 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:33:20.633 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:33:21.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:21 smithi164 ceph-mon[33167]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:21.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:21 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1024442159' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:21.467 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:33:21 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:33:21.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:21 smithi143 ceph-mon[29604]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:21.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:21 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1024442159' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:21.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:21 smithi143 ceph-mon[38451]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:21.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:21 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1024442159' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:22.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:22 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:22.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:22 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:22.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:22 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:23.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:23 smithi164 ceph-mon[33167]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:23.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:23 smithi143 ceph-mon[29604]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:23.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:23 smithi143 ceph-mon[38451]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:24.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:33:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:33:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:33:25.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:25 smithi164 ceph-mon[33167]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:25.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:25 smithi143 ceph-mon[29604]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:25.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:33:25 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:33:25] "GET /metrics HTTP/1.1" 200 219782 "" "Prometheus/2.43.0" 2024-09-11T07:33:25.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:25 smithi143 ceph-mon[38451]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:25.634 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:33:25.947 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:33:25.947 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:33:25.947 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:33:26.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:26 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2761172458' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:26.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:26.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:26 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2761172458' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:26.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:26.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:26 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2761172458' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:26.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:27.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:27 smithi164 ceph-mon[33167]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:27.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:27 smithi143 ceph-mon[29604]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:27.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:27 smithi143 ceph-mon[38451]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:29.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:29 smithi164 ceph-mon[33167]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:29.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:29 smithi143 ceph-mon[29604]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:29.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:29 smithi143 ceph-mon[38451]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:30.949 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:33:31.263 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:33:31.263 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:33:31.263 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:33:31.263 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:31 smithi143 ceph-mon[29604]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:31.263 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:31 smithi143 ceph-mon[38451]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:31.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:31 smithi164 ceph-mon[33167]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:31.467 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:33:31 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:33:32.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:32 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/700815323' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:32.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:32 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:32.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:32 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/700815323' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:32.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:32 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:32.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:32 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/700815323' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:32.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:32 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:33.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:33 smithi164 ceph-mon[33167]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:33.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:33 smithi143 ceph-mon[29604]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:33.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:33 smithi143 ceph-mon[38451]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:34.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:33:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:33:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:33:35.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:35 smithi164 ceph-mon[33167]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:35.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:35 smithi143 ceph-mon[29604]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:35.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:35 smithi143 ceph-mon[38451]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:35.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:33:35 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:33:35] "GET /metrics HTTP/1.1" 200 219783 "" "Prometheus/2.43.0" 2024-09-11T07:33:36.264 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:33:36.578 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:33:36.578 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:33:36.578 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:33:37.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:37 smithi164 ceph-mon[33167]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:37.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:37 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3903054485' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:37.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:37 smithi143 ceph-mon[29604]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:37.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:37 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3903054485' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:37.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:37 smithi143 ceph-mon[38451]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:37.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:37 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3903054485' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:39.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:39 smithi164 ceph-mon[33167]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:39.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:39 smithi143 ceph-mon[29604]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:39.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:39 smithi143 ceph-mon[38451]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:41.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:41 smithi143 ceph-mon[29604]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:41.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:41 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:41.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:41 smithi143 ceph-mon[38451]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:41.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:41 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:41.580 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:33:41.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:41 smithi164 ceph-mon[33167]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:41.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:41 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:41.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:33:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:33:41.892 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:33:41.892 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:33:41.892 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:33:42.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:42 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:42.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:42 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3131937438' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:42.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:42 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:42.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:42 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3131937438' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:42.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:42 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:42.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:42 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3131937438' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:43.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:43 smithi143 ceph-mon[29604]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:43.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:43 smithi143 ceph-mon[38451]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:43.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:43 smithi164 ceph-mon[33167]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:44.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:33:44 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:33:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:33:45.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:45 smithi143 ceph-mon[29604]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:45.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:45 smithi143 ceph-mon[38451]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:45.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:33:45 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:33:45] "GET /metrics HTTP/1.1" 200 219717 "" "Prometheus/2.43.0" 2024-09-11T07:33:45.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:45 smithi164 ceph-mon[33167]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:46.894 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:33:47.208 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:33:47.208 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:33:47.209 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:33:47.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:47 smithi143 ceph-mon[29604]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:47.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:47 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/4031247660' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:47.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:47 smithi143 ceph-mon[38451]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:47.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:47 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/4031247660' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:47.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:47 smithi164 ceph-mon[33167]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:47.726 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:47 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/4031247660' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:49.519 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:49 smithi164 ceph-mon[33167]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:49.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:49 smithi143 ceph-mon[29604]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:49.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:49 smithi143 ceph-mon[38451]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:51.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:51 smithi143 ceph-mon[29604]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:51 smithi143 ceph-mon[38451]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:51.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:51 smithi164 ceph-mon[33167]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:51.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:33:51 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:33:52.210 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:33:52.526 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:33:52.526 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:33:52.526 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:33:52.526 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:52 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:52.527 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:52 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:52.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:52 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:53.548 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:53 smithi143 ceph-mon[38451]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:53.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:53 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3413547328' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:53.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:53 smithi143 ceph-mon[29604]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:53.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:53 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3413547328' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:53.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:53 smithi164 ceph-mon[33167]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:53.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:53 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3413547328' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:54.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:54 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:33:54.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:54 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:33:54.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:54 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:33:54.717 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:33:54 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:33:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:33:55.548 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:33:55 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:33:55] "GET /metrics HTTP/1.1" 200 219717 "" "Prometheus/2.43.0" 2024-09-11T07:33:55.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:55 smithi143 ceph-mon[29604]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:55.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:55 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:33:55.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:55 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:33:55.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:55 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:33:55.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:55 smithi143 ceph-mon[38451]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:55.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:55 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:33:55.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:55 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:33:55.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:55 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:33:55.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:55 smithi164 ceph-mon[33167]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:55.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:55 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:33:55.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:55 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:33:55.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:55 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:33:56.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:56 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:56.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:56 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:56.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:56 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:57.528 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:33:57.529 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:57 smithi143 ceph-mon[29604]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:57.529 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:57 smithi143 ceph-mon[38451]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:57.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:57 smithi164 ceph-mon[33167]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:33:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:33:57.842 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:33:58.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:58 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3670176870' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:58.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:58 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3670176870' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:58.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:58 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3670176870' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:33:59.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:33:59 smithi143 ceph-mon[29604]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:59.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:33:59 smithi143 ceph-mon[38451]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:59.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:33:59 smithi164 ceph-mon[33167]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:01.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:01 smithi143 ceph-mon[29604]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:01.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:01 smithi143 ceph-mon[38451]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:01.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:01 smithi164 ceph-mon[33167]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:01.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:34:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:34:02.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:02 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:02.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:02 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:02.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:02 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:02.843 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:34:03.156 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:34:03.156 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:34:03.157 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:34:03.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:03 smithi143 ceph-mon[29604]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:03.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:03 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2712470859' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:03.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:03 smithi143 ceph-mon[38451]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:03.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:03 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2712470859' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:03.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:03 smithi164 ceph-mon[33167]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:03.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:03 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2712470859' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:04.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:34:04 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:34:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:34:05.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:05 smithi143 ceph-mon[29604]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:05.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:34:05 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:34:05] "GET /metrics HTTP/1.1" 200 219721 "" "Prometheus/2.43.0" 2024-09-11T07:34:05.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:05 smithi143 ceph-mon[38451]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:05.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:05 smithi164 ceph-mon[33167]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:07.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:07 smithi143 ceph-mon[29604]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:07.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:07 smithi143 ceph-mon[38451]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:07.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:07 smithi164 ceph-mon[33167]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:08.158 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:34:08.477 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:34:08.478 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:34:08.478 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:34:09.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:09 smithi143 ceph-mon[29604]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:09.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:09 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/704980186' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:09.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:09 smithi143 ceph-mon[38451]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:09.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:09 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/704980186' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:09.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:09 smithi164 ceph-mon[33167]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:09.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:09 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/704980186' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:11.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:11 smithi143 ceph-mon[29604]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:11.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:11 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:11.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:11 smithi143 ceph-mon[38451]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:11.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:11 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:11.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:11 smithi164 ceph-mon[33167]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:11.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:11 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:11.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:34:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:34:12.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:12 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:12.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:12 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:12.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:12 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:13.479 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:34:13.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:13 smithi143 ceph-mon[29604]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:13.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:13 smithi143 ceph-mon[38451]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:13.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:13 smithi164 ceph-mon[33167]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:13.790 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:34:13.790 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:34:13.790 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:34:14.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:14 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3218168765' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:14.717 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:34:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:34:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:34:14.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:14 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3218168765' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:14.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:14 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3218168765' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:15.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:15 smithi143 ceph-mon[29604]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:15.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:34:15 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:34:15] "GET /metrics HTTP/1.1" 200 219723 "" "Prometheus/2.43.0" 2024-09-11T07:34:15.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:15 smithi143 ceph-mon[38451]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:15.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:15 smithi164 ceph-mon[33167]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:17.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:17 smithi164 ceph-mon[33167]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:17.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:17 smithi143 ceph-mon[29604]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:17.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:17 smithi143 ceph-mon[38451]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:18.792 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:34:19.105 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:34:19.105 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:34:19.105 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:34:19.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:19 smithi143 ceph-mon[29604]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:19.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:19 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2632723463' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:19.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:19 smithi143 ceph-mon[38451]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:19.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:19 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2632723463' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:19.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:19 smithi164 ceph-mon[33167]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:19.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:19 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2632723463' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:21.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:21 smithi164 ceph-mon[33167]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:21.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:34:21 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:34:21.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:21 smithi143 ceph-mon[29604]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:21.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:21 smithi143 ceph-mon[38451]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:22.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:22 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:22.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:22 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:22.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:22 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:23.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:23 smithi164 ceph-mon[33167]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:23.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:23 smithi143 ceph-mon[29604]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:23.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:23 smithi143 ceph-mon[38451]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:24.107 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:34:24.422 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:34:24.422 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:34:24.422 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:34:24.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:34:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:34:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:34:25.548 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:34:25 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:34:25] "GET /metrics HTTP/1.1" 200 219723 "" "Prometheus/2.43.0" 2024-09-11T07:34:25.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:25 smithi143 ceph-mon[29604]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:25.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:25 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1813576446' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:25.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:25 smithi143 ceph-mon[38451]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:25.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:25 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1813576446' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:25.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:25 smithi164 ceph-mon[33167]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:25.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:25 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1813576446' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:26.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:26.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:26.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:27.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:27 smithi164 ceph-mon[33167]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:27.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:27 smithi143 ceph-mon[29604]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:27.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:27 smithi143 ceph-mon[38451]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:29.424 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:34:29.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:29 smithi164 ceph-mon[33167]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:29.735 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:34:29.735 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:34:29.735 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:34:29.736 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:29 smithi143 ceph-mon[29604]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:29.736 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:29 smithi143 ceph-mon[38451]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:30.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:30 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/966771064' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:30.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:30 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/966771064' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:30.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:30 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/966771064' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:31.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:31 smithi164 ceph-mon[33167]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:31.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:34:31 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:34:31.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:31 smithi143 ceph-mon[29604]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:31.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:31 smithi143 ceph-mon[38451]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:32.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:32 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:32.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:32 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:32.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:32 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:33.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:33 smithi164 ceph-mon[33167]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:33.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:33 smithi143 ceph-mon[29604]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:33.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:33 smithi143 ceph-mon[38451]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:34.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:34:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:34:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:34:34.737 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:34:35.049 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:34:35.050 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:34:35.050 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:34:35.356 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:34:35 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:34:35] "GET /metrics HTTP/1.1" 200 219732 "" "Prometheus/2.43.0" 2024-09-11T07:34:35.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:35 smithi164 ceph-mon[33167]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:35.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:35 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/275624322' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:35.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:35 smithi143 ceph-mon[29604]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:35.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:35 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/275624322' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:35.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:35 smithi143 ceph-mon[38451]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:35.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:35 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/275624322' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:37.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:37 smithi164 ceph-mon[33167]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:37.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:37 smithi143 ceph-mon[29604]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:37.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:37 smithi143 ceph-mon[38451]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:39.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:39 smithi164 ceph-mon[33167]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:39.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:39 smithi143 ceph-mon[29604]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:39.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:39 smithi143 ceph-mon[38451]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:40.051 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:34:40.362 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:34:40.362 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:34:40.363 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:34:40.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:40 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/213155769' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:40.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:40 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/213155769' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:40.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:40 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/213155769' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:41.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:41 smithi164 ceph-mon[33167]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:41.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:41 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:41.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:34:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:34:41.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:41 smithi143 ceph-mon[29604]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:41.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:41 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:41.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:41 smithi143 ceph-mon[38451]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:41.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:41 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:42.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:42 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:42.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:42 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:42.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:42 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:43.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:43 smithi164 ceph-mon[33167]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:43.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:43 smithi143 ceph-mon[29604]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:43.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:43 smithi143 ceph-mon[38451]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:44.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:34:44 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:34:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:34:45.365 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:34:45.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:45 smithi143 ceph-mon[29604]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:45.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:34:45 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:34:45] "GET /metrics HTTP/1.1" 200 219734 "" "Prometheus/2.43.0" 2024-09-11T07:34:45.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:45 smithi143 ceph-mon[38451]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:45.678 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:34:45.678 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:34:45.678 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:34:45.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:45 smithi164 ceph-mon[33167]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:46.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:46 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/13808712' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:46.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:46 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/13808712' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:46.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:46 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/13808712' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:47.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:47 smithi164 ceph-mon[33167]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:47.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:47 smithi143 ceph-mon[29604]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:47.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:47 smithi143 ceph-mon[38451]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:49.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:49 smithi164 ceph-mon[33167]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:49.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:49 smithi143 ceph-mon[29604]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:49.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:49 smithi143 ceph-mon[38451]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:50.680 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:34:50.990 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:34:50.990 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:34:50.990 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:34:51.716 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:34:51 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:34:51.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:51 smithi164 ceph-mon[33167]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:51.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:51 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3329854782' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:51.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:51 smithi143 ceph-mon[38451]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:51.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:51 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3329854782' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:51.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:51 smithi143 ceph-mon[29604]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:51.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:51 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3329854782' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:52.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:52 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:52.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:52 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:52.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:52 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:53.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:53 smithi164 ceph-mon[33167]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:53.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:53 smithi143 ceph-mon[29604]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:53.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:53 smithi143 ceph-mon[38451]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:54.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:34:54 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:34:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:34:55.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:34:55 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:34:55] "GET /metrics HTTP/1.1" 200 219734 "" "Prometheus/2.43.0" 2024-09-11T07:34:55.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:55 smithi143 ceph-mon[29604]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:55.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:55 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:34:55.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:55 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:34:55.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:55 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:34:55.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:55 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:34:55.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:55 smithi143 ceph-mon[38451]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:55.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:55 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:34:55.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:55 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:34:55.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:55 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:34:55.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:55 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:34:55.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:55 smithi164 ceph-mon[33167]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:55.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:55 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:34:55.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:55 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:34:55.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:55 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:34:55.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:55 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:34:55.992 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:34:56.304 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:34:56.304 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:34:56.304 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:34:56.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:56 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:56.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:56 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/735529847' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:56.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:56 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:56.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:56 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/735529847' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:56.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:56 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:56.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:56 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/735529847' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:34:57.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:57 smithi164 ceph-mon[33167]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:57.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:57 smithi143 ceph-mon[29604]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:57.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:57 smithi143 ceph-mon[38451]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:34:59.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:34:59 smithi164 ceph-mon[33167]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:59.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:34:59 smithi143 ceph-mon[29604]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:34:59.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:34:59 smithi143 ceph-mon[38451]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:01.306 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:35:01.622 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:35:01.622 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:35:01.623 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:35:01.623 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:01 smithi143 ceph-mon[29604]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:01.623 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:01 smithi143 ceph-mon[38451]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:01.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:01 smithi164 ceph-mon[33167]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:01.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:35:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:35:02.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:02 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:02.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:02 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3245375958' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:35:02.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:02 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:02.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:02 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3245375958' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:35:02.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:02 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:02.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:02 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3245375958' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:35:03.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:03 smithi164 ceph-mon[33167]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:03.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:03 smithi143 ceph-mon[29604]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:03.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:03 smithi143 ceph-mon[38451]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:04.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:35:04 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:35:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:35:05.424 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:35:05 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:35:05] "GET /metrics HTTP/1.1" 200 219735 "" "Prometheus/2.43.0" 2024-09-11T07:35:05.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:05 smithi164 ceph-mon[33167]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:05.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:05 smithi143 ceph-mon[29604]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:05.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:05 smithi143 ceph-mon[38451]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:06.624 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:35:06.936 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:35:06.936 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:35:06.936 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:35:07.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:07 smithi164 ceph-mon[33167]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:07.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:07 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/991074877' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:35:07.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:07 smithi143 ceph-mon[29604]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:07.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:07 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/991074877' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:35:07.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:07 smithi143 ceph-mon[38451]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:07.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:07 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/991074877' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:35:09.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:09 smithi164 ceph-mon[33167]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:09.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:09 smithi143 ceph-mon[29604]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:09.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:09 smithi143 ceph-mon[38451]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:11.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:11 smithi164 ceph-mon[33167]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:11.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:11 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:11.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:35:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:35:11.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:11 smithi143 ceph-mon[29604]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:11.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:11 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:11.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:11 smithi143 ceph-mon[38451]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:11.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:11 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:35:12.250 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== 2024-09-11T07:35:12.250 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== ']' 2024-09-11T07:35:12.250 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:35:12.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:12 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:12.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:12 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2879552365' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:35:12.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:12 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:12.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:12 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2879552365' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:35:12.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:12 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:12.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:12 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2879552365' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:35:13.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:13 smithi164 ceph-mon[33167]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:13.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:13 smithi143 ceph-mon[29604]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:13.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:13 smithi143 ceph-mon[38451]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:14.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:35:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:35:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:35:15.490 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:35:15 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:35:15] "GET /metrics HTTP/1.1" 200 219732 "" "Prometheus/2.43.0" 2024-09-11T07:35:15.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:15 smithi143 ceph-mon[29604]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:15.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:15 smithi143 ceph-mon[38451]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:15.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:15 smithi164 ceph-mon[33167]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:17.252 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.0 2024-09-11T07:35:17.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:17 smithi143 ceph-mon[29604]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:17.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:17 smithi143 ceph-mon[38451]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:17.567 INFO:teuthology.orchestra.run.smithi143.stdout:rotating key for osd.1 2024-09-11T07:35:17.568 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQBSR+FmWURDJRAARO1TK94qg6BcRWDbgM9nIw== 2024-09-11T07:35:17.568 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQAxRuFm8LcNKRAAeg4gUDwqljddTOVFIamzwg== == AQBSR+FmWURDJRAARO1TK94qg6BcRWDbgM9nIw== ']' 2024-09-11T07:35:17.568 INFO:teuthology.orchestra.run.smithi143.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T07:35:17.568 INFO:teuthology.orchestra.run.smithi143.stderr:+ echo 'rotating key for osd.1' 2024-09-11T07:35:17.568 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.1 2024-09-11T07:35:17.869 INFO:teuthology.orchestra.run.smithi143.stderr:+ K=AQBIRuFmfTbUMhAA+008aAtZeCTeTVl+JcSN5A== 2024-09-11T07:35:17.869 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQBIRuFmfTbUMhAA+008aAtZeCTeTVl+JcSN5A== 2024-09-11T07:35:17.869 INFO:teuthology.orchestra.run.smithi143.stderr:+ ceph orch daemon rotate-key osd.1 2024-09-11T07:35:17.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:17 smithi164 ceph-mon[33167]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:18.125 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled to rotate-key osd.1 on host 'smithi143' 2024-09-11T07:35:18.136 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQBIRuFmfTbUMhAA+008aAtZeCTeTVl+JcSN5A== == AQBIRuFmfTbUMhAA+008aAtZeCTeTVl+JcSN5A== ']' 2024-09-11T07:35:18.136 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:35:18.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3441922009' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:35:18.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/450699447' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T07:35:18.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:18.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:18.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:35:18.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:35:18.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:35:18.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:18.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-11T07:35:18.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-11T07:35:18.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3441922009' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:35:18.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/450699447' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T07:35:18.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:18.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:18.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:35:18.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:35:18.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:35:18.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:18.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-11T07:35:18.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:18 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-11T07:35:18.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:18 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3441922009' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-11T07:35:18.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:18 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/450699447' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T07:35:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:18 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:18 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:18 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:35:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:18 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:35:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:18 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:35:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:18 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:18 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-11T07:35:18.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:18 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-11T07:35:19.676 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:19 smithi143 ceph-mon[29604]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:19.677 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:19 smithi143 ceph-mon[29604]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:35:19.677 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:19 smithi143 ceph-mon[29604]: Schedule rotate-key daemon osd.1 2024-09-11T07:35:19.677 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:19 smithi143 ceph-mon[29604]: Rotating authentication key for osd.1 2024-09-11T07:35:19.677 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:19 smithi143 ceph-mon[29604]: Reconfiguring daemon osd.1 on smithi143 2024-09-11T07:35:19.677 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:19 smithi143 ceph-mon[38451]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:19.677 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:19 smithi143 ceph-mon[38451]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:35:19.677 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:19 smithi143 ceph-mon[38451]: Schedule rotate-key daemon osd.1 2024-09-11T07:35:19.677 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:19 smithi143 ceph-mon[38451]: Rotating authentication key for osd.1 2024-09-11T07:35:19.677 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:19 smithi143 ceph-mon[38451]: Reconfiguring daemon osd.1 on smithi143 2024-09-11T07:35:19.868 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:19 smithi164 ceph-mon[33167]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:19.868 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:19 smithi164 ceph-mon[33167]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:35:19.868 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:19 smithi164 ceph-mon[33167]: Schedule rotate-key daemon osd.1 2024-09-11T07:35:19.868 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:19 smithi164 ceph-mon[33167]: Rotating authentication key for osd.1 2024-09-11T07:35:19.868 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:19 smithi164 ceph-mon[33167]: Reconfiguring daemon osd.1 on smithi143 2024-09-11T07:35:21.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:20 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:21.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:20 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:21.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:20 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:21.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:20 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:21.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:20 smithi143 ceph-mon[29604]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:21.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:20 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:21.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:20 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:21.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:20 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:21.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:20 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:21.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:20 smithi143 ceph-mon[38451]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:21.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:20 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:21.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:20 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:21.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:20 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:21.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:20 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:21.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:20 smithi164 ceph-mon[33167]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:21.716 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:35:21 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:35:22.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:22 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:22.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:22 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:22.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:22 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:23.139 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.1 2024-09-11T07:35:23.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:23 smithi143 ceph-mon[29604]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:23.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:23 smithi143 ceph-mon[38451]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:23.453 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQBIRuFmfTbUMhAA+008aAtZeCTeTVl+JcSN5A== 2024-09-11T07:35:23.453 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQBIRuFmfTbUMhAA+008aAtZeCTeTVl+JcSN5A== == AQBIRuFmfTbUMhAA+008aAtZeCTeTVl+JcSN5A== ']' 2024-09-11T07:35:23.453 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:35:23.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:23 smithi164 ceph-mon[33167]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:24.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:24 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/381139296' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T07:35:24.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:24 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/381139296' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T07:35:24.396 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:24 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/381139296' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T07:35:24.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:35:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:35:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:35:25.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:25 smithi143 ceph-mon[29604]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:25.299 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:35:25 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:35:25] "GET /metrics HTTP/1.1" 200 219732 "" "Prometheus/2.43.0" 2024-09-11T07:35:25.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:25 smithi143 ceph-mon[38451]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:25.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:25 smithi164 ceph-mon[33167]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:26.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:26.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:26.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:27.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:27 smithi164 ceph-mon[33167]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:27.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:27 smithi143 ceph-mon[29604]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:27.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:27 smithi143 ceph-mon[38451]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:28.455 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.1 2024-09-11T07:35:28.767 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQBIRuFmfTbUMhAA+008aAtZeCTeTVl+JcSN5A== 2024-09-11T07:35:28.768 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQBIRuFmfTbUMhAA+008aAtZeCTeTVl+JcSN5A== == AQBIRuFmfTbUMhAA+008aAtZeCTeTVl+JcSN5A== ']' 2024-09-11T07:35:28.768 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:35:29.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:29 smithi164 ceph-mon[33167]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:29.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:29 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3571583990' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T07:35:29.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:29 smithi143 ceph-mon[29604]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:29.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:29 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3571583990' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T07:35:29.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:29 smithi143 ceph-mon[38451]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:29.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:29 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3571583990' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T07:35:31.458 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:31 smithi164 ceph-mon[33167]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:31.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:31 smithi143 ceph-mon[29604]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:31.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:31 smithi143 ceph-mon[38451]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:31.716 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:35:31 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:35:32.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:32 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:32.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:32 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:32.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:32 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:33.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:33 smithi164 ceph-mon[33167]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:33.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:33 smithi143 ceph-mon[29604]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:33.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:33 smithi143 ceph-mon[38451]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:33.769 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.1 2024-09-11T07:35:34.081 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQBIRuFmfTbUMhAA+008aAtZeCTeTVl+JcSN5A== 2024-09-11T07:35:34.082 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQBIRuFmfTbUMhAA+008aAtZeCTeTVl+JcSN5A== == AQBIRuFmfTbUMhAA+008aAtZeCTeTVl+JcSN5A== ']' 2024-09-11T07:35:34.082 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:35:34.466 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:35:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:35:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:35:34.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:34 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2777173589' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T07:35:34.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:34 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2777173589' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T07:35:34.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:34 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2777173589' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T07:35:35.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:35 smithi143 ceph-mon[29604]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:35.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:35 smithi143 ceph-mon[38451]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:35.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:35:35 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:35:35] "GET /metrics HTTP/1.1" 200 219741 "" "Prometheus/2.43.0" 2024-09-11T07:35:35.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:35 smithi164 ceph-mon[33167]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:37.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:37 smithi143 ceph-mon[29604]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:37.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:37 smithi143 ceph-mon[38451]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:37.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:37 smithi164 ceph-mon[33167]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:39.083 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.1 2024-09-11T07:35:39.393 INFO:teuthology.orchestra.run.smithi143.stdout:rotating key for osd.2 2024-09-11T07:35:39.394 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQA2SOFmXORDDBAA/ro+ha8s58TEyMtWYBuiXw== 2024-09-11T07:35:39.394 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQBIRuFmfTbUMhAA+008aAtZeCTeTVl+JcSN5A== == AQA2SOFmXORDDBAA/ro+ha8s58TEyMtWYBuiXw== ']' 2024-09-11T07:35:39.394 INFO:teuthology.orchestra.run.smithi143.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T07:35:39.394 INFO:teuthology.orchestra.run.smithi143.stderr:+ echo 'rotating key for osd.2' 2024-09-11T07:35:39.394 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:39 smithi143 ceph-mon[29604]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:39.395 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:39 smithi143 ceph-mon[38451]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:39.395 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.2 2024-09-11T07:35:39.697 INFO:teuthology.orchestra.run.smithi143.stderr:+ K=AQBfRuFm60QSExAABZVNJRmvAOOroIU4lIwDyg== 2024-09-11T07:35:39.697 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQBfRuFm60QSExAABZVNJRmvAOOroIU4lIwDyg== 2024-09-11T07:35:39.697 INFO:teuthology.orchestra.run.smithi143.stderr:+ ceph orch daemon rotate-key osd.2 2024-09-11T07:35:39.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:39 smithi164 ceph-mon[33167]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:39.949 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled to rotate-key osd.2 on host 'smithi143' 2024-09-11T07:35:39.961 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQBfRuFm60QSExAABZVNJRmvAOOroIU4lIwDyg== == AQBfRuFm60QSExAABZVNJRmvAOOroIU4lIwDyg== ']' 2024-09-11T07:35:39.961 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:35:40.548 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1566698383' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T07:35:40.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3074747552' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T07:35:40.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:40.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:40.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:35:40.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:40.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:40.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:35:40.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:35:40.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:40.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1566698383' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T07:35:40.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3074747552' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T07:35:40.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:40.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:40.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:35:40.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:40.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:40.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:35:40.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:35:40.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:40 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:40.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:40 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1566698383' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-11T07:35:40.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:40 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3074747552' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T07:35:40.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:40 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:40.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:40 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:40.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:40 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:35:40.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:40 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:40.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:40 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:40.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:40 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:35:40.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:40 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:35:40.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:40 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:41.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:41 smithi143 ceph-mon[29604]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:35:41.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:41 smithi143 ceph-mon[29604]: Schedule rotate-key daemon osd.2 2024-09-11T07:35:41.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:41 smithi143 ceph-mon[29604]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:41.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:41 smithi143 ceph-mon[29604]: Rotating authentication key for osd.2 2024-09-11T07:35:41.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:41 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-11T07:35:41.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:41 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-11T07:35:41.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:41 smithi143 ceph-mon[29604]: Reconfiguring daemon osd.2 on smithi143 2024-09-11T07:35:41.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:41 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:41.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:41 smithi143 ceph-mon[38451]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:35:41.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:41 smithi143 ceph-mon[38451]: Schedule rotate-key daemon osd.2 2024-09-11T07:35:41.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:41 smithi143 ceph-mon[38451]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:41.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:41 smithi143 ceph-mon[38451]: Rotating authentication key for osd.2 2024-09-11T07:35:41.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:41 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-11T07:35:41.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:41 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-11T07:35:41.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:41 smithi143 ceph-mon[38451]: Reconfiguring daemon osd.2 on smithi143 2024-09-11T07:35:41.300 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:41 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:41.716 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:35:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:35:41.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:41 smithi164 ceph-mon[33167]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:35:41.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:41 smithi164 ceph-mon[33167]: Schedule rotate-key daemon osd.2 2024-09-11T07:35:41.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:41 smithi164 ceph-mon[33167]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:41.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:41 smithi164 ceph-mon[33167]: Rotating authentication key for osd.2 2024-09-11T07:35:41.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:41 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-11T07:35:41.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:41 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-11T07:35:41.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:41 smithi164 ceph-mon[33167]: Reconfiguring daemon osd.2 on smithi143 2024-09-11T07:35:41.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:41 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:42.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:42 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:42.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:42 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:42.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:42 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:42.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:42 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:42.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:42 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:42.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:42 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:42.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:42 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:42.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:42 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:42.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:42 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:42.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:42 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:42.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:42 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:42.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:42 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:42.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:42 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:42.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:42 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:42.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:42 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:35:43.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:43 smithi143 ceph-mon[29604]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:43.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:43 smithi143 ceph-mon[38451]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:43.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:43 smithi164 ceph-mon[33167]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:44.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:35:44 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:35:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:35:44.963 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.2 2024-09-11T07:35:45.256 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:35:45 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:35:45] "GET /metrics HTTP/1.1" 200 219748 "" "Prometheus/2.43.0" 2024-09-11T07:35:45.278 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQBfRuFm60QSExAABZVNJRmvAOOroIU4lIwDyg== 2024-09-11T07:35:45.278 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQBfRuFm60QSExAABZVNJRmvAOOroIU4lIwDyg== == AQBfRuFm60QSExAABZVNJRmvAOOroIU4lIwDyg== ']' 2024-09-11T07:35:45.278 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:35:45.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:45 smithi143 ceph-mon[29604]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:45.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:45 smithi143 ceph-mon[38451]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:45.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:45 smithi164 ceph-mon[33167]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:46.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:46 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3089705134' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T07:35:46.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:46 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3089705134' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T07:35:46.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:46 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3089705134' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T07:35:47.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:47 smithi143 ceph-mon[29604]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:47.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:47 smithi143 ceph-mon[38451]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:47.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:47 smithi164 ceph-mon[33167]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:49.519 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:49 smithi164 ceph-mon[33167]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:49.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:49 smithi143 ceph-mon[29604]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:49.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:49 smithi143 ceph-mon[38451]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:50.280 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.2 2024-09-11T07:35:50.590 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQBfRuFm60QSExAABZVNJRmvAOOroIU4lIwDyg== 2024-09-11T07:35:50.591 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQBfRuFm60QSExAABZVNJRmvAOOroIU4lIwDyg== == AQBfRuFm60QSExAABZVNJRmvAOOroIU4lIwDyg== ']' 2024-09-11T07:35:50.591 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:35:51.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:51 smithi143 ceph-mon[29604]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:51.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:51 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1520641888' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T07:35:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:51 smithi143 ceph-mon[38451]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:51 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1520641888' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T07:35:51.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:51 smithi164 ceph-mon[33167]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:51.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:51 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1520641888' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T07:35:51.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:35:51 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:35:52.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:52 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:52.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:52 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:52.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:52 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:53.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:53 smithi143 ceph-mon[29604]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:53.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:53 smithi143 ceph-mon[38451]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:53.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:53 smithi164 ceph-mon[33167]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:54.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:35:54 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:35:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:35:55.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:55 smithi143 ceph-mon[29604]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:55.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:35:55 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:35:55] "GET /metrics HTTP/1.1" 200 219748 "" "Prometheus/2.43.0" 2024-09-11T07:35:55.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:55 smithi143 ceph-mon[38451]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:55.592 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.2 2024-09-11T07:35:55.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:55 smithi164 ceph-mon[33167]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:55.908 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQBfRuFm60QSExAABZVNJRmvAOOroIU4lIwDyg== 2024-09-11T07:35:55.908 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQBfRuFm60QSExAABZVNJRmvAOOroIU4lIwDyg== == AQBfRuFm60QSExAABZVNJRmvAOOroIU4lIwDyg== ']' 2024-09-11T07:35:55.908 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:35:56.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:56 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/432427936' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T07:35:56.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:56 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:56.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:56 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/432427936' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T07:35:56.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:56 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:56.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:56 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/432427936' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T07:35:56.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:56 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:57.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:57 smithi143 ceph-mon[29604]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:57.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:57 smithi143 ceph-mon[38451]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:57.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:57 smithi164 ceph-mon[33167]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:35:59.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:35:59 smithi143 ceph-mon[29604]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:59.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:35:59 smithi143 ceph-mon[38451]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:35:59.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:35:59 smithi164 ceph-mon[33167]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:00.910 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.2 2024-09-11T07:36:01.221 INFO:teuthology.orchestra.run.smithi143.stdout:rotating key for osd.3 2024-09-11T07:36:01.222 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQBMSOFmFP50EhAA4KHAeframrprFzmsKHvdTg== 2024-09-11T07:36:01.222 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQBfRuFm60QSExAABZVNJRmvAOOroIU4lIwDyg== == AQBMSOFmFP50EhAA4KHAeframrprFzmsKHvdTg== ']' 2024-09-11T07:36:01.222 INFO:teuthology.orchestra.run.smithi143.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T07:36:01.222 INFO:teuthology.orchestra.run.smithi143.stderr:+ echo 'rotating key for osd.3' 2024-09-11T07:36:01.222 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.3 2024-09-11T07:36:01.527 INFO:teuthology.orchestra.run.smithi143.stderr:+ K=AQB2RuFmw78yBxAAjve+Pmwmd/j0Q+TH2OIqqA== 2024-09-11T07:36:01.528 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQB2RuFmw78yBxAAjve+Pmwmd/j0Q+TH2OIqqA== 2024-09-11T07:36:01.528 INFO:teuthology.orchestra.run.smithi143.stderr:+ ceph orch daemon rotate-key osd.3 2024-09-11T07:36:01.781 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled to rotate-key osd.3 on host 'smithi143' 2024-09-11T07:36:01.781 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:01 smithi143 ceph-mon[29604]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:01.782 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:01 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1169832299' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T07:36:01.782 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:01 smithi143 ceph-mon[38451]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:01.782 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:01 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1169832299' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T07:36:01.793 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQB2RuFmw78yBxAAjve+Pmwmd/j0Q+TH2OIqqA== == AQB2RuFmw78yBxAAjve+Pmwmd/j0Q+TH2OIqqA== ']' 2024-09-11T07:36:01.793 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:36:01.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:01 smithi164 ceph-mon[33167]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:01.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:01 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1169832299' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-11T07:36:01.967 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:36:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:36:02.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:02.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/640926086' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T07:36:02.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:02.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:02.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:36:02.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:02.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:02.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:36:02.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:36:02.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:02.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:02.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/640926086' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T07:36:02.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:02.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:02.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:36:02.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:02.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:02.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:36:02.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:36:02.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:02 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:02.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:02 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:02.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:02 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/640926086' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T07:36:02.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:02 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:02.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:02 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:02.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:02 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:36:02.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:02 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:02.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:02 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:02.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:02 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:36:02.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:02 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:36:02.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:02 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:03.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:03 smithi143 ceph-mon[29604]: from='client.24980 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:36:03.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:03 smithi143 ceph-mon[29604]: Schedule rotate-key daemon osd.3 2024-09-11T07:36:03.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:03 smithi143 ceph-mon[29604]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:03.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:03 smithi143 ceph-mon[29604]: Rotating authentication key for osd.3 2024-09-11T07:36:03.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:03 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-11T07:36:03.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:03 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-11T07:36:03.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:03 smithi143 ceph-mon[29604]: Reconfiguring daemon osd.3 on smithi143 2024-09-11T07:36:03.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:03 smithi143 ceph-mon[38451]: from='client.24980 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:36:03.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:03 smithi143 ceph-mon[38451]: Schedule rotate-key daemon osd.3 2024-09-11T07:36:03.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:03 smithi143 ceph-mon[38451]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:03.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:03 smithi143 ceph-mon[38451]: Rotating authentication key for osd.3 2024-09-11T07:36:03.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:03 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-11T07:36:03.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:03 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-11T07:36:03.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:03 smithi143 ceph-mon[38451]: Reconfiguring daemon osd.3 on smithi143 2024-09-11T07:36:03.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:03 smithi164 ceph-mon[33167]: from='client.24980 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:36:03.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:03 smithi164 ceph-mon[33167]: Schedule rotate-key daemon osd.3 2024-09-11T07:36:03.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:03 smithi164 ceph-mon[33167]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:03.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:03 smithi164 ceph-mon[33167]: Rotating authentication key for osd.3 2024-09-11T07:36:03.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:03 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-11T07:36:03.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:03 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-11T07:36:03.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:03 smithi164 ceph-mon[33167]: Reconfiguring daemon osd.3 on smithi143 2024-09-11T07:36:04.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:36:04 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:36:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:36:05.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:05 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:05.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:05 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:05.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:05 smithi164 ceph-mon[33167]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T07:36:05.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:05 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:05.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:05 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:05.548 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:36:05 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:36:05] "GET /metrics HTTP/1.1" 200 219768 "" "Prometheus/2.43.0" 2024-09-11T07:36:05.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:05 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:05.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:05 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:05.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:05 smithi143 ceph-mon[29604]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T07:36:05.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:05 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:05.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:05 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:05.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:05 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:05.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:05 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:05.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:05 smithi143 ceph-mon[38451]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T07:36:05.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:05 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:05.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:05 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:06.796 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.3 2024-09-11T07:36:07.113 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQB2RuFmw78yBxAAjve+Pmwmd/j0Q+TH2OIqqA== 2024-09-11T07:36:07.113 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQB2RuFmw78yBxAAjve+Pmwmd/j0Q+TH2OIqqA== == AQB2RuFmw78yBxAAjve+Pmwmd/j0Q+TH2OIqqA== ']' 2024-09-11T07:36:07.113 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:36:07.305 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:07 smithi143 ceph-mon[29604]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T07:36:07.305 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:07 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2349307754' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T07:36:07.305 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:07 smithi143 ceph-mon[38451]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T07:36:07.305 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:07 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2349307754' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T07:36:07.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:07 smithi164 ceph-mon[33167]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T07:36:07.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:07 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2349307754' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T07:36:09.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:09 smithi164 ceph-mon[33167]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:09.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:09 smithi143 ceph-mon[29604]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:09.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:09 smithi143 ceph-mon[38451]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:11.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:11 smithi143 ceph-mon[29604]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T07:36:11.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:11 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:11.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:11 smithi143 ceph-mon[38451]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T07:36:11.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:11 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:11.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:11 smithi164 ceph-mon[33167]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T07:36:11.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:11 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:11.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:36:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:36:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.3 2024-09-11T07:36:12.433 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQB2RuFmw78yBxAAjve+Pmwmd/j0Q+TH2OIqqA== 2024-09-11T07:36:12.434 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQB2RuFmw78yBxAAjve+Pmwmd/j0Q+TH2OIqqA== == AQB2RuFmw78yBxAAjve+Pmwmd/j0Q+TH2OIqqA== ']' 2024-09-11T07:36:12.434 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:36:12.434 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:12 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:12.434 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:12 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:12.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:12 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:13.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:13 smithi143 ceph-mon[29604]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:13.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:13 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/649820539' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T07:36:13.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:13 smithi143 ceph-mon[38451]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:13.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:13 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/649820539' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T07:36:13.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:13 smithi164 ceph-mon[33167]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:13.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:13 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/649820539' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T07:36:14.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:36:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:36:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:36:15.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:15 smithi143 ceph-mon[29604]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T07:36:15.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:15 smithi143 ceph-mon[38451]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T07:36:15.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:36:15 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:36:15] "GET /metrics HTTP/1.1" 200 219773 "" "Prometheus/2.43.0" 2024-09-11T07:36:15.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:15 smithi164 ceph-mon[33167]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T07:36:17.435 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.3 2024-09-11T07:36:17.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:17 smithi143 ceph-mon[29604]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:17.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:17 smithi143 ceph-mon[38451]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:17.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:17 smithi164 ceph-mon[33167]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:17.750 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQB2RuFmw78yBxAAjve+Pmwmd/j0Q+TH2OIqqA== 2024-09-11T07:36:17.750 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQB2RuFmw78yBxAAjve+Pmwmd/j0Q+TH2OIqqA== == AQB2RuFmw78yBxAAjve+Pmwmd/j0Q+TH2OIqqA== ']' 2024-09-11T07:36:17.750 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:36:18.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:18 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2775640491' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T07:36:18.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:18 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2775640491' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T07:36:18.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:18 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2775640491' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T07:36:19.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:19 smithi143 ceph-mon[29604]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:19.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:19 smithi143 ceph-mon[38451]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:19.620 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:19 smithi164 ceph-mon[33167]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:21.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:21 smithi143 ceph-mon[29604]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:21.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:21 smithi143 ceph-mon[38451]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:21.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:21 smithi164 ceph-mon[33167]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:21.966 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:36:21 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:36:22.752 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.3 2024-09-11T07:36:22.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:22 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:22.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:22 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:22.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:22 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:23.064 INFO:teuthology.orchestra.run.smithi143.stdout:rotating key for osd.4 2024-09-11T07:36:23.065 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQBiSOFm6xi3HhAABdIeFVsCYs8KGmOh2g9GTg== 2024-09-11T07:36:23.065 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQB2RuFmw78yBxAAjve+Pmwmd/j0Q+TH2OIqqA== == AQBiSOFm6xi3HhAABdIeFVsCYs8KGmOh2g9GTg== ']' 2024-09-11T07:36:23.065 INFO:teuthology.orchestra.run.smithi143.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T07:36:23.065 INFO:teuthology.orchestra.run.smithi143.stderr:+ echo 'rotating key for osd.4' 2024-09-11T07:36:23.065 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.4 2024-09-11T07:36:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:+ K=AQCLRuFm9TqtKhAAE9KfJU8Uj22ROaqeHFUPQA== 2024-09-11T07:36:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQCLRuFm9TqtKhAAE9KfJU8Uj22ROaqeHFUPQA== 2024-09-11T07:36:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:+ ceph orch daemon rotate-key osd.4 2024-09-11T07:36:23.624 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled to rotate-key osd.4 on host 'smithi164' 2024-09-11T07:36:23.625 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:23 smithi143 ceph-mon[29604]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:23.625 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:23 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1688574710' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T07:36:23.625 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:23 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2716108450' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T07:36:23.625 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:23 smithi143 ceph-mon[38451]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:23.625 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:23 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1688574710' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T07:36:23.626 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:23 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2716108450' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T07:36:23.636 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQCLRuFm9TqtKhAAE9KfJU8Uj22ROaqeHFUPQA== == AQCLRuFm9TqtKhAAE9KfJU8Uj22ROaqeHFUPQA== ']' 2024-09-11T07:36:23.636 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:36:23.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:23 smithi164 ceph-mon[33167]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:23.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:23 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1688574710' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-11T07:36:23.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:23 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2716108450' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T07:36:24.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:36:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:36:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:36:24.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:24 smithi164 ceph-mon[33167]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:36:24.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:24 smithi164 ceph-mon[33167]: Schedule rotate-key daemon osd.4 2024-09-11T07:36:24.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:24 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:24.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:24 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:24.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:24 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:36:24.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:24 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:36:24.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:24 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:36:24.718 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:24 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:24.718 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:24 smithi164 ceph-mon[33167]: Rotating authentication key for osd.4 2024-09-11T07:36:24.718 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:24 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-11T07:36:24.718 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:24 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-11T07:36:24.718 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:24 smithi164 ceph-mon[33167]: Reconfiguring daemon osd.4 on smithi164 2024-09-11T07:36:24.718 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:24 smithi164 ceph-mon[33167]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:25.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[29604]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:36:25.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[29604]: Schedule rotate-key daemon osd.4 2024-09-11T07:36:25.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:25.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:25.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:36:25.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:36:25.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:36:25.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:25.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[29604]: Rotating authentication key for osd.4 2024-09-11T07:36:25.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-11T07:36:25.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-11T07:36:25.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[29604]: Reconfiguring daemon osd.4 on smithi164 2024-09-11T07:36:25.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[29604]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:25.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[38451]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:36:25.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[38451]: Schedule rotate-key daemon osd.4 2024-09-11T07:36:25.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:25.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:25.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:36:25.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:36:25.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:36:25.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:25.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[38451]: Rotating authentication key for osd.4 2024-09-11T07:36:25.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-11T07:36:25.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-11T07:36:25.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[38451]: Reconfiguring daemon osd.4 on smithi164 2024-09-11T07:36:25.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:24 smithi143 ceph-mon[38451]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:25.548 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:36:25 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:36:25] "GET /metrics HTTP/1.1" 200 219773 "" "Prometheus/2.43.0" 2024-09-11T07:36:26.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:26.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:26.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:26.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:26.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:26.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:26.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:26.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:26.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:26.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:26.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:26.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:26.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:26.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:26.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:27.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:27 smithi164 ceph-mon[33167]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:27.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:27 smithi143 ceph-mon[29604]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:27.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:27 smithi143 ceph-mon[38451]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:28.638 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.4 2024-09-11T07:36:28.948 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQCLRuFm9TqtKhAAE9KfJU8Uj22ROaqeHFUPQA== 2024-09-11T07:36:28.948 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQCLRuFm9TqtKhAAE9KfJU8Uj22ROaqeHFUPQA== == AQCLRuFm9TqtKhAAE9KfJU8Uj22ROaqeHFUPQA== ']' 2024-09-11T07:36:28.948 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:36:29.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:29 smithi164 ceph-mon[33167]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:29.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:29 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3404575320' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T07:36:29.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:29 smithi143 ceph-mon[29604]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:29.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:29 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3404575320' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T07:36:29.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:29 smithi143 ceph-mon[38451]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:29.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:29 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3404575320' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T07:36:31.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:31 smithi164 ceph-mon[33167]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:31.716 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:36:31 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:36:31.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:31 smithi143 ceph-mon[29604]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:31.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:31 smithi143 ceph-mon[38451]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:32.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:32 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:32.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:32 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:32.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:32 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:33.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:33 smithi164 ceph-mon[33167]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:33.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:33 smithi143 ceph-mon[29604]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:33.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:33 smithi143 ceph-mon[38451]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:33.950 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.4 2024-09-11T07:36:34.264 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQCLRuFm9TqtKhAAE9KfJU8Uj22ROaqeHFUPQA== 2024-09-11T07:36:34.264 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQCLRuFm9TqtKhAAE9KfJU8Uj22ROaqeHFUPQA== == AQCLRuFm9TqtKhAAE9KfJU8Uj22ROaqeHFUPQA== ']' 2024-09-11T07:36:34.264 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:36:34.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:34 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2332905019' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T07:36:34.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:34 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2332905019' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T07:36:34.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:34 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2332905019' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T07:36:34.717 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:36:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:36:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:36:35.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:35 smithi143 ceph-mon[29604]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:35.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:36:35 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:36:35] "GET /metrics HTTP/1.1" 200 219762 "" "Prometheus/2.43.0" 2024-09-11T07:36:35.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:35 smithi143 ceph-mon[38451]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:35.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:35 smithi164 ceph-mon[33167]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:37.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:37 smithi164 ceph-mon[33167]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:37.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:37 smithi143 ceph-mon[29604]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:37.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:37 smithi143 ceph-mon[38451]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:39.265 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.4 2024-09-11T07:36:39.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:39 smithi143 ceph-mon[29604]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:39.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:39 smithi143 ceph-mon[38451]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:39.582 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQCLRuFm9TqtKhAAE9KfJU8Uj22ROaqeHFUPQA== 2024-09-11T07:36:39.582 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQCLRuFm9TqtKhAAE9KfJU8Uj22ROaqeHFUPQA== == AQCLRuFm9TqtKhAAE9KfJU8Uj22ROaqeHFUPQA== ']' 2024-09-11T07:36:39.582 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:36:39.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:39 smithi164 ceph-mon[33167]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:40.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:40 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1622403718' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T07:36:40.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:40 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1622403718' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T07:36:40.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:40 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1622403718' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T07:36:41.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:41 smithi164 ceph-mon[33167]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:41.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:41 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:41.717 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:36:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:36:41.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:41 smithi143 ceph-mon[29604]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:41.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:41 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:41.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:41 smithi143 ceph-mon[38451]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:41.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:41 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:42.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:42 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:42.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:42 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:42.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:42 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:43.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:43 smithi164 ceph-mon[33167]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:43.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:43 smithi143 ceph-mon[29604]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:43.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:43 smithi143 ceph-mon[38451]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:44.584 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.4 2024-09-11T07:36:44.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:36:44 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:36:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:36:44.895 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQB3SOFmljM/KhAABMUVSjKrynaW0QPWpv3l8w== 2024-09-11T07:36:44.895 INFO:teuthology.orchestra.run.smithi143.stdout:rotating key for osd.5 2024-09-11T07:36:44.895 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQCLRuFm9TqtKhAAE9KfJU8Uj22ROaqeHFUPQA== == AQB3SOFmljM/KhAABMUVSjKrynaW0QPWpv3l8w== ']' 2024-09-11T07:36:44.896 INFO:teuthology.orchestra.run.smithi143.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T07:36:44.896 INFO:teuthology.orchestra.run.smithi143.stderr:+ echo 'rotating key for osd.5' 2024-09-11T07:36:44.896 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.5 2024-09-11T07:36:45.197 INFO:teuthology.orchestra.run.smithi143.stderr:+ K=AQCiRuFmGfc2NBAA9KilZFwrREFt3Q1XZbtgaA== 2024-09-11T07:36:45.197 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQCiRuFmGfc2NBAA9KilZFwrREFt3Q1XZbtgaA== 2024-09-11T07:36:45.197 INFO:teuthology.orchestra.run.smithi143.stderr:+ ceph orch daemon rotate-key osd.5 2024-09-11T07:36:45.451 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled to rotate-key osd.5 on host 'smithi164' 2024-09-11T07:36:45.451 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:36:45 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:36:45] "GET /metrics HTTP/1.1" 200 219762 "" "Prometheus/2.43.0" 2024-09-11T07:36:45.452 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:45 smithi143 ceph-mon[29604]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:45.452 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:45 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3847602805' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T07:36:45.452 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:45 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/4205279255' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T07:36:45.452 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:45 smithi143 ceph-mon[38451]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:45.452 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:45 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3847602805' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T07:36:45.452 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:45 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/4205279255' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T07:36:45.463 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQCiRuFmGfc2NBAA9KilZFwrREFt3Q1XZbtgaA== == AQCiRuFmGfc2NBAA9KilZFwrREFt3Q1XZbtgaA== ']' 2024-09-11T07:36:45.464 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:36:45.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:45 smithi164 ceph-mon[33167]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:45.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:45 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3847602805' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-11T07:36:45.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:45 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/4205279255' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T07:36:46.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:46 smithi164 ceph-mon[33167]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:36:46.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:46 smithi164 ceph-mon[33167]: Schedule rotate-key daemon osd.5 2024-09-11T07:36:46.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:46 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:46.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:46 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:46.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:46 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:36:46.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:46 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:36:46.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:46 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:36:46.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:46 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:46.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:46 smithi164 ceph-mon[33167]: Rotating authentication key for osd.5 2024-09-11T07:36:46.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:46 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-11T07:36:46.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:46 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-11T07:36:46.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:46 smithi164 ceph-mon[33167]: Reconfiguring daemon osd.5 on smithi164 2024-09-11T07:36:46.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[29604]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:36:46.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[29604]: Schedule rotate-key daemon osd.5 2024-09-11T07:36:46.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:46.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:46.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:36:46.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:36:46.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:36:46.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:46.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[29604]: Rotating authentication key for osd.5 2024-09-11T07:36:46.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-11T07:36:46.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-11T07:36:46.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[29604]: Reconfiguring daemon osd.5 on smithi164 2024-09-11T07:36:46.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[38451]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:36:46.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[38451]: Schedule rotate-key daemon osd.5 2024-09-11T07:36:46.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:46.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:46.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:36:46.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:36:46.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:36:46.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:46.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[38451]: Rotating authentication key for osd.5 2024-09-11T07:36:46.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-11T07:36:46.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-11T07:36:46.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:46 smithi143 ceph-mon[38451]: Reconfiguring daemon osd.5 on smithi164 2024-09-11T07:36:47.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:47 smithi143 ceph-mon[29604]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:47.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:47 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:47.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:47 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:47.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:47 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:47.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:47 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:47.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:47 smithi143 ceph-mon[38451]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:47.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:47 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:47.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:47 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:47.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:47 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:47.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:47 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:47.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:47 smithi164 ceph-mon[33167]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:47.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:47 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:47.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:47 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:47.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:47 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:47.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:47 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:36:49.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:49 smithi143 ceph-mon[29604]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:49.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:49 smithi143 ceph-mon[38451]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:49.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:49 smithi164 ceph-mon[33167]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.5 2024-09-11T07:36:50.778 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQCiRuFmGfc2NBAA9KilZFwrREFt3Q1XZbtgaA== 2024-09-11T07:36:50.779 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQCiRuFmGfc2NBAA9KilZFwrREFt3Q1XZbtgaA== == AQCiRuFmGfc2NBAA9KilZFwrREFt3Q1XZbtgaA== ']' 2024-09-11T07:36:50.779 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:36:51.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:51 smithi143 ceph-mon[29604]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:51.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:51 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3408344863' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T07:36:51.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:51 smithi143 ceph-mon[38451]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:51.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:51 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3408344863' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T07:36:51.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:51 smithi164 ceph-mon[33167]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:51.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:51 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3408344863' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T07:36:51.967 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:36:51 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:36:52.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:52 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:52.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:52 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:52.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:52 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:53.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:53 smithi164 ceph-mon[33167]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:54.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:53 smithi143 ceph-mon[29604]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:54.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:53 smithi143 ceph-mon[38451]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:36:54.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:36:54 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:36:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:36:55.548 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:36:55 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:36:55] "GET /metrics HTTP/1.1" 200 219762 "" "Prometheus/2.43.0" 2024-09-11T07:36:55.781 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.5 2024-09-11T07:36:55.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:55 smithi164 ceph-mon[33167]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:56.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:55 smithi143 ceph-mon[29604]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:56.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:55 smithi143 ceph-mon[38451]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:56.095 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQCiRuFmGfc2NBAA9KilZFwrREFt3Q1XZbtgaA== 2024-09-11T07:36:56.096 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQCiRuFmGfc2NBAA9KilZFwrREFt3Q1XZbtgaA== == AQCiRuFmGfc2NBAA9KilZFwrREFt3Q1XZbtgaA== ']' 2024-09-11T07:36:56.096 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:36:56.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:56 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:56.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:56 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2137829320' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T07:36:57.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:56 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:57.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:56 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2137829320' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T07:36:57.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:56 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:57.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:56 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2137829320' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T07:36:57.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:57 smithi164 ceph-mon[33167]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:58.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:57 smithi143 ceph-mon[29604]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:58.048 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:57 smithi143 ceph-mon[38451]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:36:59.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:36:59 smithi164 ceph-mon[33167]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:00.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:36:59 smithi143 ceph-mon[29604]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:00.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:36:59 smithi143 ceph-mon[38451]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:01.097 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.5 2024-09-11T07:37:01.410 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQCiRuFmGfc2NBAA9KilZFwrREFt3Q1XZbtgaA== 2024-09-11T07:37:01.410 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQCiRuFmGfc2NBAA9KilZFwrREFt3Q1XZbtgaA== == AQCiRuFmGfc2NBAA9KilZFwrREFt3Q1XZbtgaA== ']' 2024-09-11T07:37:01.410 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:37:01.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:01 smithi143 ceph-mon[29604]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:01.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:01 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1385684864' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T07:37:01.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:01 smithi143 ceph-mon[38451]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:01.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:01 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1385684864' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T07:37:01.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:01 smithi164 ceph-mon[33167]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:01.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:01 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1385684864' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T07:37:01.967 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:37:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:37:02.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:02 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:03.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:02 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:03.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:02 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:03.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:03 smithi164 ceph-mon[33167]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:04.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:03 smithi143 ceph-mon[29604]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:04.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:03 smithi143 ceph-mon[38451]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:04.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:04 smithi164 ceph-mon[33167]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:04.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:37:04 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:37:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:37:05.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:04 smithi143 ceph-mon[29604]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:05.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:04 smithi143 ceph-mon[38451]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:05.548 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:05 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:37:05] "GET /metrics HTTP/1.1" 200 219772 "" "Prometheus/2.43.0" 2024-09-11T07:37:06.412 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.5 2024-09-11T07:37:06.726 INFO:teuthology.orchestra.run.smithi143.stdout:rotating key for osd.6 2024-09-11T07:37:06.727 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQCNSOFm19+0HxAA1nWmTsuYep9o9BZwbOLR2Q== 2024-09-11T07:37:06.727 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQCiRuFmGfc2NBAA9KilZFwrREFt3Q1XZbtgaA== == AQCNSOFm19+0HxAA1nWmTsuYep9o9BZwbOLR2Q== ']' 2024-09-11T07:37:06.727 INFO:teuthology.orchestra.run.smithi143.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T07:37:06.727 INFO:teuthology.orchestra.run.smithi143.stderr:+ echo 'rotating key for osd.6' 2024-09-11T07:37:06.727 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.6 2024-09-11T07:37:07.028 INFO:teuthology.orchestra.run.smithi143.stderr:+ K=AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== 2024-09-11T07:37:07.028 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== 2024-09-11T07:37:07.028 INFO:teuthology.orchestra.run.smithi143.stderr:+ ceph orch daemon rotate-key osd.6 2024-09-11T07:37:07.282 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled to rotate-key osd.6 on host 'smithi164' 2024-09-11T07:37:07.282 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:07 smithi143 ceph-mon[29604]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:07.282 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:07 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/43937843' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T07:37:07.282 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:07 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3260715631' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:07.282 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:07 smithi143 ceph-mon[38451]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:07.282 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:07 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/43937843' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T07:37:07.283 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:07 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3260715631' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== == AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== ']' 2024-09-11T07:37:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:37:07.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:07 smithi164 ceph-mon[33167]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:07.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:07 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/43937843' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-11T07:37:07.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:07 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3260715631' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:08.376 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:08 smithi164 ceph-mon[33167]: from='client.25124 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:37:08.377 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:08 smithi164 ceph-mon[33167]: Schedule rotate-key daemon osd.6 2024-09-11T07:37:08.377 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:08 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:08.377 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:08 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:08.377 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:08 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:37:08.377 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:08 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:37:08.377 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:08 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:37:08.377 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:08 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:08.377 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:08 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-11T07:37:08.377 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:08 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-11T07:37:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[29604]: from='client.25124 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:37:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[29604]: Schedule rotate-key daemon osd.6 2024-09-11T07:37:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:37:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:37:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:37:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-11T07:37:08.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-11T07:37:08.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[38451]: from='client.25124 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:37:08.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[38451]: Schedule rotate-key daemon osd.6 2024-09-11T07:37:08.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:08.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:08.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:37:08.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:37:08.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:37:08.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:08.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-11T07:37:08.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:08 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-11T07:37:09.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:09 smithi164 ceph-mon[33167]: Rotating authentication key for osd.6 2024-09-11T07:37:09.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:09 smithi164 ceph-mon[33167]: Reconfiguring daemon osd.6 on smithi164 2024-09-11T07:37:09.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:09 smithi164 ceph-mon[33167]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:09.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:09 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:09.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:09 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:09.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:09 smithi143 ceph-mon[29604]: Rotating authentication key for osd.6 2024-09-11T07:37:09.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:09 smithi143 ceph-mon[29604]: Reconfiguring daemon osd.6 on smithi164 2024-09-11T07:37:09.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:09 smithi143 ceph-mon[29604]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:09.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:09 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:09.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:09 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:09.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:09 smithi143 ceph-mon[38451]: Rotating authentication key for osd.6 2024-09-11T07:37:09.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:09 smithi143 ceph-mon[38451]: Reconfiguring daemon osd.6 on smithi164 2024-09-11T07:37:09.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:09 smithi143 ceph-mon[38451]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:09.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:09 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:09.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:09 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:10.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:10 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:10.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:10 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:10.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:10 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:10.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:10 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:10.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:10 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:10.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:10 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:11.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:11 smithi143 ceph-mon[29604]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:11.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:11 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:11.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:11 smithi143 ceph-mon[38451]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:11.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:11 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:11.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:11 smithi164 ceph-mon[33167]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:11.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:11 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:11.967 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:37:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:37:12.295 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.6 2024-09-11T07:37:12.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:12 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:12.548 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:12 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:12.613 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== 2024-09-11T07:37:12.613 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== == AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== ']' 2024-09-11T07:37:12.613 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:37:12.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:12 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:13.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:13 smithi143 ceph-mon[29604]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:13.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:13 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/200232847' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:13.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:13 smithi143 ceph-mon[38451]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:13.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:13 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/200232847' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:13.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:13 smithi164 ceph-mon[33167]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:13.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:13 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/200232847' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:14.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:37:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:37:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:37:15.506 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:15 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:37:15] "GET /metrics HTTP/1.1" 200 219783 "" "Prometheus/2.43.0" 2024-09-11T07:37:15.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:15 smithi143 ceph-mon[29604]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:15.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:15 smithi143 ceph-mon[38451]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:15.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:15 smithi164 ceph-mon[33167]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:17.615 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.6 2024-09-11T07:37:17.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:17 smithi143 ceph-mon[29604]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:17.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:17 smithi143 ceph-mon[38451]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:17.931 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== 2024-09-11T07:37:17.931 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== == AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== ']' 2024-09-11T07:37:17.931 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:37:17.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:17 smithi164 ceph-mon[33167]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:18.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:18 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2189272570' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:18.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:18 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2189272570' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:18.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:18 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2189272570' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:19.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:19 smithi143 ceph-mon[29604]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:19.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:19 smithi143 ceph-mon[38451]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:19.868 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:19 smithi164 ceph-mon[33167]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:21.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:21 smithi143 ceph-mon[29604]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:21.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:21 smithi143 ceph-mon[38451]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:21.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:21 smithi164 ceph-mon[33167]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:21.967 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:37:21 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:37:22.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:22 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:22.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:22 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:22.933 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.6 2024-09-11T07:37:22.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:22 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:23.244 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== 2024-09-11T07:37:23.245 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== == AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== ']' 2024-09-11T07:37:23.245 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:37:23.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:23 smithi143 ceph-mon[29604]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:23.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:23 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/4140273548' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:23.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:23 smithi143 ceph-mon[38451]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:23.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:23 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/4140273548' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:23.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:23 smithi164 ceph-mon[33167]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:23.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:23 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/4140273548' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:24.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:37:24 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:37:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:37:25.527 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:25 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:37:25] "GET /metrics HTTP/1.1" 200 219783 "" "Prometheus/2.43.0" 2024-09-11T07:37:25.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:25 smithi143 ceph-mon[29604]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:25.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:25 smithi143 ceph-mon[38451]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:25.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:25 smithi164 ceph-mon[33167]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:26.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:26 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:26.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:26 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:26.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:26 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:27.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:27 smithi143 ceph-mon[29604]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:27.804 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:27 smithi143 ceph-mon[38451]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:27.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:27 smithi164 ceph-mon[33167]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:28.246 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.6 2024-09-11T07:37:28.556 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== 2024-09-11T07:37:28.556 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== == AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== ']' 2024-09-11T07:37:28.557 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:37:29.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:29 smithi143 ceph-mon[29604]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:29.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:29 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/466667348' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:29.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:29 smithi143 ceph-mon[38451]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:29.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:29 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/466667348' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:29.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:29 smithi164 ceph-mon[33167]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:29.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:29 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/466667348' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:31.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:31 smithi143 ceph-mon[29604]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:31.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:31 smithi143 ceph-mon[38451]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:31.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:31 smithi164 ceph-mon[33167]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:31.967 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:37:31 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:37:32.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:32 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:33.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:32 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:33.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:32 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:33.558 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.6 2024-09-11T07:37:33.869 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQCjSOFmrn5qLBAAckadb6ByqG5g06sSZtbTmA== 2024-09-11T07:37:33.870 INFO:teuthology.orchestra.run.smithi143.stdout:rotating key for osd.7 2024-09-11T07:37:33.870 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:33 smithi143 ceph-mon[29604]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:33.871 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQC5RuFmhZ4cKBAAomKrc2UUvxi6niLnIEblWA== == AQCjSOFmrn5qLBAAckadb6ByqG5g06sSZtbTmA== ']' 2024-09-11T07:37:33.871 INFO:teuthology.orchestra.run.smithi143.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T07:37:33.871 INFO:teuthology.orchestra.run.smithi143.stderr:+ echo 'rotating key for osd.7' 2024-09-11T07:37:33.871 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:33 smithi143 ceph-mon[38451]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:33.871 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.7 2024-09-11T07:37:33.966 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:33 smithi164 ceph-mon[33167]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:34.171 INFO:teuthology.orchestra.run.smithi143.stderr:+ K=AQDPRuFmfhPCNRAAPmTJttjmLPAFXybDvoMVcQ== 2024-09-11T07:37:34.171 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQDPRuFmfhPCNRAAPmTJttjmLPAFXybDvoMVcQ== 2024-09-11T07:37:34.171 INFO:teuthology.orchestra.run.smithi143.stderr:+ ceph orch daemon rotate-key osd.7 2024-09-11T07:37:34.422 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled to rotate-key osd.7 on host 'smithi164' 2024-09-11T07:37:34.433 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQDPRuFmfhPCNRAAPmTJttjmLPAFXybDvoMVcQ== == AQDPRuFmfhPCNRAAPmTJttjmLPAFXybDvoMVcQ== ']' 2024-09-11T07:37:34.434 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:37:34.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:37:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:37:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:37:34.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:34 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2051082063' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:34.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:34 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/783816310' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T07:37:34.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:34 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:34.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:34 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:34.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:34 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:37:34.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:34 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:37:34.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:34 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:37:34.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:34 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:34.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:34 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-11T07:37:34.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:34 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-11T07:37:34.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2051082063' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:34.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/783816310' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T07:37:34.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:34.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:34.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:37:34.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:37:34.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:37:34.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:34.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-11T07:37:34.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-11T07:37:34.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2051082063' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-11T07:37:34.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/783816310' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T07:37:34.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:34.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:34.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:37:34.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:37:34.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:37:34.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:34.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-11T07:37:34.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:34 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-11T07:37:35.548 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:35 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:37:35] "GET /metrics HTTP/1.1" 200 219782 "" "Prometheus/2.43.0" 2024-09-11T07:37:35.860 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:35 smithi164 ceph-mon[33167]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:35.860 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:35 smithi164 ceph-mon[33167]: from='client.25178 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:37:35.860 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:35 smithi164 ceph-mon[33167]: Schedule rotate-key daemon osd.7 2024-09-11T07:37:35.860 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:35 smithi164 ceph-mon[33167]: Rotating authentication key for osd.7 2024-09-11T07:37:35.860 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:35 smithi164 ceph-mon[33167]: Reconfiguring daemon osd.7 on smithi164 2024-09-11T07:37:36.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:35 smithi143 ceph-mon[29604]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:36.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:35 smithi143 ceph-mon[29604]: from='client.25178 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:37:36.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:35 smithi143 ceph-mon[29604]: Schedule rotate-key daemon osd.7 2024-09-11T07:37:36.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:35 smithi143 ceph-mon[29604]: Rotating authentication key for osd.7 2024-09-11T07:37:36.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:35 smithi143 ceph-mon[29604]: Reconfiguring daemon osd.7 on smithi164 2024-09-11T07:37:36.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:35 smithi143 ceph-mon[38451]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:36.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:35 smithi143 ceph-mon[38451]: from='client.25178 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:37:36.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:35 smithi143 ceph-mon[38451]: Schedule rotate-key daemon osd.7 2024-09-11T07:37:36.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:35 smithi143 ceph-mon[38451]: Rotating authentication key for osd.7 2024-09-11T07:37:36.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:35 smithi143 ceph-mon[38451]: Reconfiguring daemon osd.7 on smithi164 2024-09-11T07:37:37.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:36 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:37.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:36 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:37.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:36 smithi164 ceph-mon[33167]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:37.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:36 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:37.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:36 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:37.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:36 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:37.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:36 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:37.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:36 smithi143 ceph-mon[29604]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:37.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:36 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:37.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:36 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:37.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:36 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:37.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:36 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:37.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:36 smithi143 ceph-mon[38451]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:37.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:36 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:37.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:36 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:39.436 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.7 2024-09-11T07:37:39.437 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:39 smithi143 ceph-mon[29604]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:39.437 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:39 smithi143 ceph-mon[38451]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:39.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:39 smithi164 ceph-mon[33167]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:39.746 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQDPRuFmfhPCNRAAPmTJttjmLPAFXybDvoMVcQ== 2024-09-11T07:37:39.746 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQDPRuFmfhPCNRAAPmTJttjmLPAFXybDvoMVcQ== == AQDPRuFmfhPCNRAAPmTJttjmLPAFXybDvoMVcQ== ']' 2024-09-11T07:37:39.746 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:37:40.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:40 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/466738743' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T07:37:40.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:40 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/466738743' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T07:37:40.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:40 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/466738743' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T07:37:41.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:41 smithi164 ceph-mon[33167]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:41.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:41 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:41.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:41 smithi143 ceph-mon[29604]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:41.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:41 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:41.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:41 smithi143 ceph-mon[38451]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:41.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:41 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:41.966 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:37:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:37:42.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:42 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:42.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:42 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:42.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:42 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:43.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:43 smithi164 ceph-mon[33167]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:43.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:43 smithi143 ceph-mon[29604]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:43.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:43 smithi143 ceph-mon[38451]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:44.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:37:44 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:37:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:37:44.748 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.7 2024-09-11T07:37:45.060 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQDPRuFmfhPCNRAAPmTJttjmLPAFXybDvoMVcQ== 2024-09-11T07:37:45.060 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQDPRuFmfhPCNRAAPmTJttjmLPAFXybDvoMVcQ== == AQDPRuFmfhPCNRAAPmTJttjmLPAFXybDvoMVcQ== ']' 2024-09-11T07:37:45.060 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:37:45.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:45 smithi164 ceph-mon[33167]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:45.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:45 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1695446325' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T07:37:45.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:45 smithi143 ceph-mon[29604]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:45.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:45 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1695446325' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T07:37:45.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:45 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:37:45] "GET /metrics HTTP/1.1" 200 219782 "" "Prometheus/2.43.0" 2024-09-11T07:37:45.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:45 smithi143 ceph-mon[38451]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:45.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:45 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1695446325' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T07:37:47.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:47 smithi164 ceph-mon[33167]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:47.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:47 smithi143 ceph-mon[29604]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:47.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:47 smithi143 ceph-mon[38451]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:49.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:49 smithi164 ceph-mon[33167]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:49.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:49 smithi143 ceph-mon[29604]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:49.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:49 smithi143 ceph-mon[38451]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:50.062 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.7 2024-09-11T07:37:50.376 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQDPRuFmfhPCNRAAPmTJttjmLPAFXybDvoMVcQ== 2024-09-11T07:37:50.377 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQDPRuFmfhPCNRAAPmTJttjmLPAFXybDvoMVcQ== == AQDPRuFmfhPCNRAAPmTJttjmLPAFXybDvoMVcQ== ']' 2024-09-11T07:37:50.377 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:37:51.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:51 smithi164 ceph-mon[33167]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:51.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:51 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1824733661' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T07:37:51.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:51 smithi143 ceph-mon[29604]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:51.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:51 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1824733661' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T07:37:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:51 smithi143 ceph-mon[38451]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:51.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:51 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1824733661' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T07:37:51.966 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:37:51 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:37:52.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:52 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:52.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:52 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:52.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:52 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:53.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:53 smithi164 ceph-mon[33167]: pgmap v228: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:53.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:53 smithi143 ceph-mon[29604]: pgmap v228: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:53.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:53 smithi143 ceph-mon[38451]: pgmap v228: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:54.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:37:54 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:37:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:37:55.378 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key osd.7 2024-09-11T07:37:55.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:55 smithi164 ceph-mon[33167]: pgmap v229: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:55.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:55 smithi143 ceph-mon[29604]: pgmap v229: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:55.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:55 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:37:55] "GET /metrics HTTP/1.1" 200 219782 "" "Prometheus/2.43.0" 2024-09-11T07:37:55.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:55 smithi143 ceph-mon[38451]: pgmap v229: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:55.690 INFO:teuthology.orchestra.run.smithi143.stdout:rotating key for mgr.y 2024-09-11T07:37:55.690 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQC+SOFmBe8CHhAAflwOWsYCYA4T0dpddhe7rQ== 2024-09-11T07:37:55.690 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQDPRuFmfhPCNRAAPmTJttjmLPAFXybDvoMVcQ== == AQC+SOFmBe8CHhAAflwOWsYCYA4T0dpddhe7rQ== ']' 2024-09-11T07:37:55.690 INFO:teuthology.orchestra.run.smithi143.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T07:37:55.690 INFO:teuthology.orchestra.run.smithi143.stderr:+ echo 'rotating key for mgr.y' 2024-09-11T07:37:55.690 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key mgr.y 2024-09-11T07:37:55.990 INFO:teuthology.orchestra.run.smithi143.stderr:+ K=AQC4ReFm9OfQBBAAdBNGEsf+oSUoKort95h6fQ== 2024-09-11T07:37:55.990 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQC4ReFm9OfQBBAAdBNGEsf+oSUoKort95h6fQ== 2024-09-11T07:37:55.990 INFO:teuthology.orchestra.run.smithi143.stderr:+ ceph orch daemon rotate-key mgr.y 2024-09-11T07:37:56.250 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled to rotate-key mgr.y on host 'smithi143' 2024-09-11T07:37:56.250 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:56 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1066837021' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T07:37:56.250 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:56 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1779058142' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T07:37:56.251 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:56 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:56.251 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:56 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1066837021' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T07:37:56.251 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:56 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1779058142' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T07:37:56.251 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:56 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:56.262 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQC4ReFm9OfQBBAAdBNGEsf+oSUoKort95h6fQ== == AQC4ReFm9OfQBBAAdBNGEsf+oSUoKort95h6fQ== ']' 2024-09-11T07:37:56.262 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:37:56.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:56 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1066837021' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-11T07:37:56.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:56 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1779058142' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T07:37:56.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:56 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:57.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[29604]: pgmap v230: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:57.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[29604]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:37:57.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[29604]: Schedule rotate-key daemon mgr.y 2024-09-11T07:37:57.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:57.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:57.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:37:57.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:37:57.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:37:57.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:57.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[29604]: Rotating authentication key for mgr.y 2024-09-11T07:37:57.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T07:37:57.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T07:37:57.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:37:57.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:37:57.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:37:57.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[29604]: Reconfiguring daemon mgr.y on smithi143 2024-09-11T07:37:57.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[38451]: pgmap v230: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:57.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[38451]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:37:57.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[38451]: Schedule rotate-key daemon mgr.y 2024-09-11T07:37:57.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:57.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:57.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:37:57.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:37:57.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:37:57.551 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:57.552 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[38451]: Rotating authentication key for mgr.y 2024-09-11T07:37:57.552 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T07:37:57.552 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T07:37:57.552 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:37:57.552 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:37:57.552 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:37:57.552 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-mon[38451]: Reconfiguring daemon mgr.y on smithi143 2024-09-11T07:37:57.716 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-mon[33167]: pgmap v230: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:57.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-mon[33167]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:37:57.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-mon[33167]: Schedule rotate-key daemon mgr.y 2024-09-11T07:37:57.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:57.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:57.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:37:57.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:37:57.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:37:57.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:57.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-mon[33167]: Rotating authentication key for mgr.y 2024-09-11T07:37:57.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T07:37:57.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T07:37:57.717 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:37:57.718 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:37:57.718 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:37:57.718 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-mon[33167]: Reconfiguring daemon mgr.y on smithi143 2024-09-11T07:37:58.119 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.972+0000 7ff146a27640 -1 mgr.server reply reply (13) Permission denied access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication 2024-09-11T07:37:58.119 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.974+0000 7ff13818a640 -1 log_channel(cephadm) log [ERR] : Non-zero return from ['ceph', '-k', '/var/lib/ceph/mgr/ceph-y/keyring', '-n', 'mgr.y', 'tell', 'mgr.y', 'rotate-key', '-i', '-']: 2024-09-11T07:37:57.968+0000 7f0129d9b640 1 -- 172.21.15.143:0/3100291457 <== mon.2 v2:172.21.15.143:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7f0114003710 con 0x7f012c106d00 2024-09-11T07:37:58.119 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.968+0000 7f0129d9b640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 2206816236031297633 != 18374858748799134293 2024-09-11T07:37:58.120 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.968+0000 7f0131184640 1 -- 172.21.15.143:0/3100291457 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f012c106d00 msgr2=0x7f012c10d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:37:58.120 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.968+0000 7f0131184640 1 --2- 172.21.15.143:0/3100291457 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f012c106d00 0x7f012c10d5a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0124009cd0 tx=0x7f0124004d10 comp rx=0 tx=0).stop 2024-09-11T07:37:58.120 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.968+0000 7f0131184640 1 -- 172.21.15.143:0/3100291457 shutdown_connections 2024-09-11T07:37:58.120 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.968+0000 7f0131184640 1 --2- 172.21.15.143:0/3100291457 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f012c106d00 0x7f012c10d5a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T07:37:58.120 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.968+0000 7f0131184640 1 --2- 172.21.15.143:0/3100291457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f012c106350 0x7f012c1067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T07:37:58.120 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.968+0000 7f0131184640 1 --2- 172.21.15.143:0/3100291457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f012c104fe0 0x7f012c105410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T07:37:58.120 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.968+0000 7f0131184640 1 -- 172.21.15.143:0/3100291457 >> 172.21.15.143:0/3100291457 conn(0x7f012c06b9f0 msgr2=0x7f012c06de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:37:58.120 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.968+0000 7f0131184640 1 -- 172.21.15.143:0/3100291457 shutdown_connections 2024-09-11T07:37:58.120 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.968+0000 7f0131184640 1 -- 172.21.15.143:0/3100291457 wait complete. 2024-09-11T07:37:58.121 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f0131184640 1 Processor -- start 2024-09-11T07:37:58.121 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f0131184640 1 -- start start 2024-09-11T07:37:58.121 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f0131184640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f012c104fe0 0x7f012c115300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T07:37:58.121 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f0131184640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f012c106350 0x7f012c115840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T07:37:58.121 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f0131184640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f012c106d00 0x7f012c11c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T07:37:58.121 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f0131184640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c11ce60 con 0x7f012c106350 2024-09-11T07:37:58.121 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f0131184640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c11cfd0 con 0x7f012c104fe0 2024-09-11T07:37:58.121 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f0131184640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012c11d190 con 0x7f012c106d00 2024-09-11T07:37:58.121 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f012b59e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f012c106d00 0x7f012c11c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T07:37:58.121 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f012a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f012c106350 0x7f012c115840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T07:37:58.121 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f012b59e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f012c106d00 0x7f012c11c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:35642/0 (socket says 172.21.15.143:35642) 2024-09-11T07:37:58.121 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f012b59e640 1 -- 172.21.15.143:0/2405476571 learned_addr learned my addr 172.21.15.143:0/2405476571 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-11T07:37:58.122 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f012ad9d640 1 --2- 172.21.15.143:0/2405476571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f012c104fe0 0x7f012c115300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T07:37:58.122 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f012a59c640 1 -- 172.21.15.143:0/2405476571 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f012c106d00 msgr2=0x7f012c11c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:37:58.122 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f012a59c640 1 --2- 172.21.15.143:0/2405476571 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f012c106d00 0x7f012c11c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T07:37:58.122 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f012a59c640 1 -- 172.21.15.143:0/2405476571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f012c104fe0 msgr2=0x7f012c115300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:37:58.122 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f012a59c640 1 --2- 172.21.15.143:0/2405476571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f012c104fe0 0x7f012c115300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T07:37:58.122 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f012a59c640 1 -- 172.21.15.143:0/2405476571 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c005040 con 0x7f012c106350 2024-09-11T07:37:58.122 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f012b59e640 1 --2- 172.21.15.143:0/2405476571 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f012c106d00 0x7f012c11c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T07:37:58.122 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f012a59c640 1 --2- 172.21.15.143:0/2405476571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f012c106350 0x7f012c115840 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f01200028a0 tx=0x7f0120009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T07:37:58.122 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f01137fe640 1 -- 172.21.15.143:0/2405476571 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120007c30 con 0x7f012c106350 2024-09-11T07:37:58.122 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.969+0000 7f01137fe640 1 -- 172.21.15.143:0/2405476571 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- auth(proto 2 2 bytes epoch 0) v1 -- 0x7f01140032d0 con 0x7f012c106350 2024-09-11T07:37:58.122 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.970+0000 7f0131184640 1 -- 172.21.15.143:0/2405476571 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01240098b0 con 0x7f012c106350 2024-09-11T07:37:58.122 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.970+0000 7f01137fe640 1 -- 172.21.15.143:0/2405476571 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(38 keys) v1 ==== 1752+0+0 (secure 0 0 0) 0x7f012000f040 con 0x7f012c106350 2024-09-11T07:37:58.123 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.970+0000 7f01137fe640 1 -- 172.21.15.143:0/2405476571 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120013820 con 0x7f012c106350 2024-09-11T07:37:58.123 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.970+0000 7f0131184640 1 -- 172.21.15.143:0/2405476571 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012c11d7e0 con 0x7f012c106350 2024-09-11T07:37:58.123 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.970+0000 7f01137fe640 1 -- 172.21.15.143:0/2405476571 <== mon.0 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7f012c11d7e0 con 0x7f012c106350 2024-09-11T07:37:58.123 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.970+0000 7f01137fe640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 2206816236031297633 != 18374858748799134293 2024-09-11T07:37:58.123 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.971+0000 7f01137fe640 1 -- 172.21.15.143:0/2405476571 <== mon.0 v2:172.21.15.143:3300/0 5 ==== mgrmap(e 22) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f0120023030 con 0x7f012c106350 2024-09-11T07:37:58.123 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.971+0000 7f01137fe640 1 --2- 172.21.15.143:0/2405476571 >> v2:172.21.15.143:6800/3574421342 conn(0x7f011406f1c0 0x7f0114071670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T07:37:58.123 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.971+0000 7f01137fe640 1 -- 172.21.15.143:0/2405476571 --> v2:172.21.15.143:6800/3574421342 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0114071d40 con 0x7f011406f1c0 2024-09-11T07:37:58.123 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.971+0000 7f01137fe640 1 -- 172.21.15.143:0/2405476571 <== mon.0 v2:172.21.15.143:3300/0 6 ==== osd_map(64..64 src has 1..64) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f012008ec90 con 0x7f012c106350 2024-09-11T07:37:58.123 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.971+0000 7f012ad9d640 1 --2- 172.21.15.143:0/2405476571 >> v2:172.21.15.143:6800/3574421342 conn(0x7f011406f1c0 0x7f0114071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T07:37:58.123 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.972+0000 7f012ad9d640 1 --2- 172.21.15.143:0/2405476571 >> v2:172.21.15.143:6800/3574421342 conn(0x7f011406f1c0 0x7f0114071670 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f012c116880 tx=0x7f011c00b040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T07:37:58.123 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:57.972+0000 7f01137fe640 1 -- 172.21.15.143:0/2405476571 <== mgr.24431 v2:172.21.15.143:6800/3574421342 1 ==== command_reply(tid 0: -13 access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication) v1 ==== 134+0+0 (secure 0 0 0) 0x7f0114071d40 con 0x7f011406f1c0 2024-09-11T07:37:58.123 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: Error EACCES: problem getting command descriptions from mgr.y 2024-09-11T07:37:58.123 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:57 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:58.123 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ignoring --setuser ceph since I am not root 2024-09-11T07:37:58.124 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ignoring --setgroup ceph since I am not root 2024-09-11T07:37:58.124 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mgr[29878]: -- 172.21.15.143:0/1378373393 <== mon.2 v2:172.21.15.143:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a0be7ba340 con 0x55a0be7c6c00 2024-09-11T07:37:58.124 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mgr[29878]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 2206816236031297633 != 18374858748799134293 2024-09-11T07:37:58.124 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:58.116+0000 7f58b0b9eec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:37:58.412 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:37:57 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: [11/Sep/2024:07:37:57] ENGINE Bus STOPPING 2024-09-11T07:37:58.498 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:58.171+0000 7f58b0b9eec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:37:58.498 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:58.236+0000 7f58b0b9eec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:37:58.498 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:58.358+0000 7f58b0b9eec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:37:58.499 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:58.496+0000 7f58b0b9eec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:37:58.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: [11/Sep/2024:07:37:58] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:37:58.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: [11/Sep/2024:07:37:58] ENGINE Bus STOPPED 2024-09-11T07:37:58.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: [11/Sep/2024:07:37:58] ENGINE Bus STARTING 2024-09-11T07:37:58.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: [11/Sep/2024:07:37:58] ENGINE Serving on http://:::9283 2024-09-11T07:37:58.716 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: [11/Sep/2024:07:37:58] ENGINE Bus STARTED 2024-09-11T07:37:59.048 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:58.757+0000 7f58b0b9eec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:37:59.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:58.821+0000 7f58b0b9eec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:37:59.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:58.996+0000 7f58b0b9eec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:37:59.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:59.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:59.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-11T07:37:59.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: Activating manager daemon x 2024-09-11T07:37:59.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T07:37:59.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-11T07:37:59.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: mgrmap e23: x(active, starting, since 0.00629555s) 2024-09-11T07:37:59.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:37:59.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:37:59.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:37:59.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:37:59.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:37:59.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:37:59.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:37:59.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:37:59.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:37:59.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:37:59.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:37:59.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:37:59.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:37:59.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:37:59.050 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:37:59.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: Manager daemon x is now available 2024-09-11T07:37:59.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:37:59.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:59.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:37:59.051 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T07:37:59.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:59.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:59.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-11T07:37:59.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: Activating manager daemon x 2024-09-11T07:37:59.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T07:37:59.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-11T07:37:59.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: mgrmap e23: x(active, starting, since 0.00629555s) 2024-09-11T07:37:59.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:37:59.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:37:59.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:37:59.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:37:59.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:37:59.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:37:59.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:37:59.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:37:59.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:37:59.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:37:59.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:37:59.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:37:59.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:37:59.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:37:59.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:37:59.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: Manager daemon x is now available 2024-09-11T07:37:59.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:37:59.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:59.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:37:59.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:58 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T07:37:59.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:59.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' 2024-09-11T07:37:59.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-11T07:37:59.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: Activating manager daemon x 2024-09-11T07:37:59.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T07:37:59.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24431 172.21.15.143:0/1172766080' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-11T07:37:59.217 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: mgrmap e23: x(active, starting, since 0.00629555s) 2024-09-11T07:37:59.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:37:59.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:37:59.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:37:59.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:37:59.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:37:59.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:37:59.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:37:59.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:37:59.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:37:59.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:37:59.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:37:59.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:37:59.218 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:37:59.219 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:37:59.219 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:37:59.219 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: Manager daemon x is now available 2024-09-11T07:37:59.219 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:37:59.219 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:59.219 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:37:59.219 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:58 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T07:38:00.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:59 smithi143 ceph-mon[29604]: mgrmap e24: x(active, since 1.00906s) 2024-09-11T07:38:00.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:37:59 smithi143 ceph-mon[29604]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:38:00.299 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:59 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:59.903+0000 7f58b0b9eec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:38:00.299 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:37:59 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:37:59.987+0000 7f58b0b9eec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:38:00.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:59 smithi143 ceph-mon[38451]: mgrmap e24: x(active, since 1.00906s) 2024-09-11T07:38:00.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:37:59 smithi143 ceph-mon[38451]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:38:00.431 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:59 smithi164 ceph-mon[33167]: mgrmap e24: x(active, since 1.00906s) 2024-09-11T07:38:00.432 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:37:59 smithi164 ceph-mon[33167]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:38:00.432 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:00.047+0000 7f8ba7f61640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:07:38:00] ENGINE Error in HTTPServer.serve 2024-09-11T07:38:00.432 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: Traceback (most recent call last): 2024-09-11T07:38:00.432 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:38:00.432 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: self._connections.run(self.expiration_interval) 2024-09-11T07:38:00.432 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:38:00.432 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: self._run(expiration_interval) 2024-09-11T07:38:00.432 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:38:00.433 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:38:00.433 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:38:00.433 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:38:00.433 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:38:00.433 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: s = self.context.wrap_socket( 2024-09-11T07:38:00.433 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:38:00.433 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: return self.sslsocket_class._create( 2024-09-11T07:38:00.433 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:38:00.433 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: self.do_handshake() 2024-09-11T07:38:00.433 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:38:00.433 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: self._sslobj.do_handshake() 2024-09-11T07:38:00.433 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:38:00.433 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:01.048 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:00 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:38:00.737+0000 7f58b0b9eec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:38:01.048 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:00 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:38:00.848+0000 7f58b0b9eec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:38:01.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:00 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:38:00.981+0000 7f58b0b9eec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:38:01.264 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key mgr.y 2024-09-11T07:38:01.327 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:38:01.055+0000 7f58b0b9eec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:38:01.327 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:38:01.180+0000 7f58b0b9eec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:38:01.327 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:38:01.262+0000 7f58b0b9eec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:38:01.327 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:38:01.325+0000 7f58b0b9eec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:38:01.328 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-mon[38451]: [11/Sep/2024:07:37:59] ENGINE Bus STARTING 2024-09-11T07:38:01.328 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-mon[38451]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:38:01.328 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-mon[38451]: [11/Sep/2024:07:38:00] ENGINE Serving on https://172.21.15.164:7150 2024-09-11T07:38:01.328 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-mon[38451]: [11/Sep/2024:07:38:00] ENGINE Bus STARTED 2024-09-11T07:38:01.328 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-mon[38451]: [11/Sep/2024:07:38:00] ENGINE Error in HTTPServer.serve 2024-09-11T07:38:01.328 INFO:journalctl@ceph.mon.c.smithi143.stdout: Traceback (most recent call last): 2024-09-11T07:38:01.328 INFO:journalctl@ceph.mon.c.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:38:01.328 INFO:journalctl@ceph.mon.c.smithi143.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:38:01.328 INFO:journalctl@ceph.mon.c.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:38:01.328 INFO:journalctl@ceph.mon.c.smithi143.stdout: self._run(expiration_interval) 2024-09-11T07:38:01.328 INFO:journalctl@ceph.mon.c.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:38:01.328 INFO:journalctl@ceph.mon.c.smithi143.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:38:01.328 INFO:journalctl@ceph.mon.c.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:38:01.329 INFO:journalctl@ceph.mon.c.smithi143.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:38:01.329 INFO:journalctl@ceph.mon.c.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:38:01.329 INFO:journalctl@ceph.mon.c.smithi143.stdout: s = self.context.wrap_socket( 2024-09-11T07:38:01.329 INFO:journalctl@ceph.mon.c.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:38:01.329 INFO:journalctl@ceph.mon.c.smithi143.stdout: return self.sslsocket_class._create( 2024-09-11T07:38:01.329 INFO:journalctl@ceph.mon.c.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:38:01.329 INFO:journalctl@ceph.mon.c.smithi143.stdout: self.do_handshake() 2024-09-11T07:38:01.329 INFO:journalctl@ceph.mon.c.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:38:01.329 INFO:journalctl@ceph.mon.c.smithi143.stdout: self._sslobj.do_handshake() 2024-09-11T07:38:01.329 INFO:journalctl@ceph.mon.c.smithi143.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:38:01.329 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:01.329 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-mon[38451]: mgrmap e25: x(active, since 2s) 2024-09-11T07:38:01.330 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-mon[29604]: [11/Sep/2024:07:37:59] ENGINE Bus STARTING 2024-09-11T07:38:01.330 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-mon[29604]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:38:01.330 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-mon[29604]: [11/Sep/2024:07:38:00] ENGINE Serving on https://172.21.15.164:7150 2024-09-11T07:38:01.330 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-mon[29604]: [11/Sep/2024:07:38:00] ENGINE Bus STARTED 2024-09-11T07:38:01.330 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-mon[29604]: [11/Sep/2024:07:38:00] ENGINE Error in HTTPServer.serve 2024-09-11T07:38:01.330 INFO:journalctl@ceph.mon.a.smithi143.stdout: Traceback (most recent call last): 2024-09-11T07:38:01.330 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:38:01.330 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:38:01.330 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:38:01.330 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._run(expiration_interval) 2024-09-11T07:38:01.330 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:38:01.330 INFO:journalctl@ceph.mon.a.smithi143.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:38:01.330 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:38:01.330 INFO:journalctl@ceph.mon.a.smithi143.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:38:01.331 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:38:01.331 INFO:journalctl@ceph.mon.a.smithi143.stdout: s = self.context.wrap_socket( 2024-09-11T07:38:01.331 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:38:01.331 INFO:journalctl@ceph.mon.a.smithi143.stdout: return self.sslsocket_class._create( 2024-09-11T07:38:01.331 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:38:01.331 INFO:journalctl@ceph.mon.a.smithi143.stdout: self.do_handshake() 2024-09-11T07:38:01.331 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:38:01.331 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._sslobj.do_handshake() 2024-09-11T07:38:01.331 INFO:journalctl@ceph.mon.a.smithi143.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:38:01.331 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:01.331 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-mon[29604]: mgrmap e25: x(active, since 2s) 2024-09-11T07:38:01.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:01 smithi164 ceph-mon[33167]: [11/Sep/2024:07:37:59] ENGINE Bus STARTING 2024-09-11T07:38:01.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:01 smithi164 ceph-mon[33167]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:38:01.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:01 smithi164 ceph-mon[33167]: [11/Sep/2024:07:38:00] ENGINE Serving on https://172.21.15.164:7150 2024-09-11T07:38:01.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:01 smithi164 ceph-mon[33167]: [11/Sep/2024:07:38:00] ENGINE Bus STARTED 2024-09-11T07:38:01.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:01 smithi164 ceph-mon[33167]: [11/Sep/2024:07:38:00] ENGINE Error in HTTPServer.serve 2024-09-11T07:38:01.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: Traceback (most recent call last): 2024-09-11T07:38:01.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:38:01.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:38:01.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:38:01.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: self._run(expiration_interval) 2024-09-11T07:38:01.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:38:01.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:38:01.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:38:01.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:38:01.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:38:01.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: s = self.context.wrap_socket( 2024-09-11T07:38:01.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:38:01.467 INFO:journalctl@ceph.mon.b.smithi164.stdout: return self.sslsocket_class._create( 2024-09-11T07:38:01.468 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:38:01.468 INFO:journalctl@ceph.mon.b.smithi164.stdout: self.do_handshake() 2024-09-11T07:38:01.468 INFO:journalctl@ceph.mon.b.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:38:01.468 INFO:journalctl@ceph.mon.b.smithi164.stdout: self._sslobj.do_handshake() 2024-09-11T07:38:01.468 INFO:journalctl@ceph.mon.b.smithi164.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:38:01.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:01 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:01.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:01 smithi164 ceph-mon[33167]: mgrmap e25: x(active, since 2s) 2024-09-11T07:38:01.586 INFO:teuthology.orchestra.run.smithi143.stdout:rotating key for mgr.x 2024-09-11T07:38:01.587 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQDUSOFmfX66ExAAFeqS7l06Pn+EZz9lj6qDMg== 2024-09-11T07:38:01.587 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQC4ReFm9OfQBBAAdBNGEsf+oSUoKort95h6fQ== == AQDUSOFmfX66ExAAFeqS7l06Pn+EZz9lj6qDMg== ']' 2024-09-11T07:38:01.587 INFO:teuthology.orchestra.run.smithi143.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-11T07:38:01.587 INFO:teuthology.orchestra.run.smithi143.stderr:+ echo 'rotating key for mgr.x' 2024-09-11T07:38:01.587 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:38:01.470+0000 7f58b0b9eec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:38:01.588 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:38:01.535+0000 7f58b0b9eec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:38:01.588 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key mgr.x 2024-09-11T07:38:01.881 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:38:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:38:01.910 INFO:teuthology.orchestra.run.smithi143.stderr:+ K=AQApRuFmWI7XAxAAyTmYHRBvchxRRPeyvRwBCg== 2024-09-11T07:38:01.910 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQApRuFmWI7XAxAAyTmYHRBvchxRRPeyvRwBCg== 2024-09-11T07:38:01.910 INFO:teuthology.orchestra.run.smithi143.stderr:+ ceph orch daemon rotate-key mgr.x 2024-09-11T07:38:01.911 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:38:01.603+0000 7f58b0b9eec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:38:01.911 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:38:01.691+0000 7f58b0b9eec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:38:01.911 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:38:01.750+0000 7f58b0b9eec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:38:01.911 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:01 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:38:01.810+0000 7f58b0b9eec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:38:02.203 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled to rotate-key mgr.x on host 'smithi164' 2024-09-11T07:38:02.203 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/3604429946' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T07:38:02.203 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-mon[38451]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:02.203 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:02.204 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:02.204 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1047958278' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T07:38:02.204 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/3604429946' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T07:38:02.204 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-mon[29604]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:02.204 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:02.204 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:02.204 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1047958278' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T07:38:02.214 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQApRuFmWI7XAxAAyTmYHRBvchxRRPeyvRwBCg== == AQApRuFmWI7XAxAAyTmYHRBvchxRRPeyvRwBCg== ']' 2024-09-11T07:38:02.214 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:38:02.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:02 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/3604429946' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-11T07:38:02.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:02 smithi164 ceph-mon[33167]: from='client.24398 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:02.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:02 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:02.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:02 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:02.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:02 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1047958278' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T07:38:03.048 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:38:02.609+0000 7f58b0b9eec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:38:03.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:38:02] ENGINE Bus STARTING 2024-09-11T07:38:03.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: CherryPy Checker: 2024-09-11T07:38:03.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: The Application mounted at '' has an empty config. 2024-09-11T07:38:03.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: 2024-09-11T07:38:03.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:38:02] ENGINE Serving on http://:::9283 2024-09-11T07:38:03.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:02 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: [11/Sep/2024:07:38:02] ENGINE Bus STARTED 2024-09-11T07:38:03.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:38:03.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:38:03.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: Schedule rotate-key daemon mgr.x 2024-09-11T07:38:03.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:03.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:03.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:03.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:03.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: Adjusting osd_memory_target on smithi143 to 3248M 2024-09-11T07:38:03.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: Standby manager daemon y started 2024-09-11T07:38:03.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='mgr.? 172.21.15.143:0/1932642879' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T07:38:03.467 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='mgr.? 172.21.15.143:0/1932642879' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:38:03.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='mgr.? 172.21.15.143:0/1932642879' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T07:38:03.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='mgr.? 172.21.15.143:0/1932642879' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:38:03.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: mgrmap e26: x(active, since 4s), standbys: y 2024-09-11T07:38:03.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:38:03.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.468 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:03 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:38:03.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:38:03.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: Schedule rotate-key daemon mgr.x 2024-09-11T07:38:03.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:03.549 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: Adjusting osd_memory_target on smithi143 to 3248M 2024-09-11T07:38:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: Standby manager daemon y started 2024-09-11T07:38:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='mgr.? 172.21.15.143:0/1932642879' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T07:38:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='mgr.? 172.21.15.143:0/1932642879' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:38:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='mgr.? 172.21.15.143:0/1932642879' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T07:38:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='mgr.? 172.21.15.143:0/1932642879' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:38:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: mgrmap e26: x(active, since 4s), standbys: y 2024-09-11T07:38:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:38:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.550 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:38:03.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:38:03.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: Schedule rotate-key daemon mgr.x 2024-09-11T07:38:03.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:03.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:03.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:03.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:03.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: Adjusting osd_memory_target on smithi143 to 3248M 2024-09-11T07:38:03.552 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.552 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: Standby manager daemon y started 2024-09-11T07:38:03.552 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='mgr.? 172.21.15.143:0/1932642879' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T07:38:03.552 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='mgr.? 172.21.15.143:0/1932642879' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:38:03.552 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='mgr.? 172.21.15.143:0/1932642879' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T07:38:03.552 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='mgr.? 172.21.15.143:0/1932642879' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:38:03.552 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: mgrmap e26: x(active, since 4s), standbys: y 2024-09-11T07:38:03.552 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:38:03.552 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:03.552 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:03 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:04.714 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:38:04] "GET /metrics HTTP/1.1" 200 34931 "" "Prometheus/2.43.0" 2024-09-11T07:38:04.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:04.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:04.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:04.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:04.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:04.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:04.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: Adjusting osd_memory_target on smithi164 to 3248M 2024-09-11T07:38:04.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:04.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:38:04.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:38:04.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: Updating smithi143:/etc/ceph/ceph.conf 2024-09-11T07:38:04.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-11T07:38:04.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:38:04.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:38:04.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:38:04.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:38:04.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:38:04.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:38:04.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:38:04.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:04.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:04.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:04.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:04.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:04.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T07:38:04.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T07:38:04.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:38:04.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:38:04.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:04 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:38:05.014 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:05.015 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:05.015 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:05.015 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:05.015 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:05.015 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:05.015 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: Adjusting osd_memory_target on smithi164 to 3248M 2024-09-11T07:38:05.015 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:05.015 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:38:05.015 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:38:05.015 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: Updating smithi143:/etc/ceph/ceph.conf 2024-09-11T07:38:05.015 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-11T07:38:05.015 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:38:05.015 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:38:05.016 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:38:05.016 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:38:05.016 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:38:05.016 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:38:05.016 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:38:05.016 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:05.016 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:05.016 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:05.016 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:05.016 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:05.016 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T07:38:05.016 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T07:38:05.016 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:38:05.016 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:38:05.017 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:38:05.017 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:05.017 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:05.017 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:05.017 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:05.017 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:05.017 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:38:05.017 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: Adjusting osd_memory_target on smithi164 to 3248M 2024-09-11T07:38:05.017 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:05.017 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:38:05.017 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:38:05.017 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: Updating smithi143:/etc/ceph/ceph.conf 2024-09-11T07:38:05.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-11T07:38:05.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:38:05.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:38:05.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:38:05.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:38:05.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:38:05.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: Updating smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:38:05.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: Updating smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:38:05.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:05.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:05.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:05.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:05.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:05.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-11T07:38:05.018 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-11T07:38:05.019 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:38:05.019 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:38:05.019 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:04 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:38:05.299 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:05 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[29855]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:38:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:38:06.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:05 smithi143 ceph-mon[29604]: Rotating authentication key for mgr.y 2024-09-11T07:38:06.048 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:05 smithi143 ceph-mon[29604]: Reconfiguring daemon mgr.y on smithi143 2024-09-11T07:38:06.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:05 smithi143 ceph-mon[38451]: Rotating authentication key for mgr.y 2024-09-11T07:38:06.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:05 smithi143 ceph-mon[38451]: Reconfiguring daemon mgr.y on smithi143 2024-09-11T07:38:06.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:05 smithi164 ceph-mon[33167]: Rotating authentication key for mgr.y 2024-09-11T07:38:06.216 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:05 smithi164 ceph-mon[33167]: Reconfiguring daemon mgr.y on smithi143 2024-09-11T07:38:07.048 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:06 smithi143 systemd[1]: Stopping Ceph mgr.y for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:38:07.216 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key mgr.x 2024-09-11T07:38:07.382 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:07 smithi143 podman[83176]: 2024-09-11 07:38:07.087509777 +0000 UTC m=+0.148838901 container died 413fb9952d95053b1ffe3fbd385beb1cf14bc0ef46ce518f4ae5405fed319330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T07:38:07.382 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:07 smithi143 ceph-mon[29604]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T07:38:07.382 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:07 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:07.382 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:07 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:07.382 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:07 smithi143 ceph-mon[38451]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T07:38:07.383 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:07 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:07.383 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:07 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:07.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:07 smithi164 ceph-mon[33167]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T07:38:07.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:07 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:07.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:07 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:07.528 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQApRuFmWI7XAxAAyTmYHRBvchxRRPeyvRwBCg== 2024-09-11T07:38:07.528 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQApRuFmWI7XAxAAyTmYHRBvchxRRPeyvRwBCg== == AQApRuFmWI7XAxAAyTmYHRBvchxRRPeyvRwBCg== ']' 2024-09-11T07:38:07.528 INFO:teuthology.orchestra.run.smithi143.stderr:+ sleep 5 2024-09-11T07:38:07.716 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:07 smithi143 podman[83176]: 2024-09-11 07:38:07.4960384 +0000 UTC m=+0.557367514 container cleanup 413fb9952d95053b1ffe3fbd385beb1cf14bc0ef46ce518f4ae5405fed319330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T07:38:07.716 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:07 smithi143 bash[83176]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y 2024-09-11T07:38:07.967 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:07 smithi143 podman[83196]: 2024-09-11 07:38:07.714805535 +0000 UTC m=+0.631853230 container remove 413fb9952d95053b1ffe3fbd385beb1cf14bc0ef46ce518f4ae5405fed319330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-11T07:38:07.967 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:07 smithi143 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-11T07:38:08.298 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:08 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/2555403934' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T07:38:08.299 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:08 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/2555403934' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T07:38:08.299 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:07 smithi143 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-11T07:38:08.299 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:07 smithi143 systemd[1]: Stopped Ceph mgr.y for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:38:08.299 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:07 smithi143 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mgr.y.service: Consumed 43.723s CPU time. 2024-09-11T07:38:08.299 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:07 smithi143 systemd[1]: Starting Ceph mgr.y for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:38:08.299 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:08 smithi143 podman[83317]: 2024-09-11 07:38:08.076789023 +0000 UTC m=+0.015494850 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:38:08.299 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:08 smithi143 podman[83317]: 2024-09-11 07:38:08.190493812 +0000 UTC m=+0.129199639 container create 49cfafd26c01840087d25af36172d22994dd823dac336dd824685d0c9066c581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-11T07:38:08.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:08 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/2555403934' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T07:38:08.798 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:08 smithi143 podman[83317]: 2024-09-11 07:38:08.435331616 +0000 UTC m=+0.374037448 container init 49cfafd26c01840087d25af36172d22994dd823dac336dd824685d0c9066c581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:38:08.798 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:08 smithi143 podman[83317]: 2024-09-11 07:38:08.440257145 +0000 UTC m=+0.378962973 container start 49cfafd26c01840087d25af36172d22994dd823dac336dd824685d0c9066c581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T07:38:08.798 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:08 smithi143 ceph-mgr[83355]: -- 172.21.15.143:0/1262962599 <== mon.2 v2:172.21.15.143:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c1e0b2e340 con 0x55c1e0b3ac00 2024-09-11T07:38:08.798 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:08 smithi143 ceph-mgr[83355]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 15698896164731679774 != 18374858748799134293 2024-09-11T07:38:08.799 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:08 smithi143 bash[83317]: 49cfafd26c01840087d25af36172d22994dd823dac336dd824685d0c9066c581 2024-09-11T07:38:08.799 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:08 smithi143 systemd[1]: Started Ceph mgr.y for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:38:08.799 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:08 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:08.553+0000 7fa24a00dec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:38:08.799 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:08 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:08.611+0000 7fa24a00dec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:38:08.799 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:08 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:08.674+0000 7fa24a00dec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:38:08.799 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:08 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:08.796+0000 7fa24a00dec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:38:09.048 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:08 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:08.930+0000 7fa24a00dec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:38:09.316 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:09.194+0000 7fa24a00dec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:38:09.316 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:09.257+0000 7fa24a00dec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:38:09.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[29604]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T07:38:09.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:09.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:09.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[29604]: Rotating authentication key for mgr.x 2024-09-11T07:38:09.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-11T07:38:09.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-11T07:38:09.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:38:09.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:38:09.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:38:09.800 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:09.428+0000 7fa24a00dec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:38:09.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[38451]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T07:38:09.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:09.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:09.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[38451]: Rotating authentication key for mgr.x 2024-09-11T07:38:09.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-11T07:38:09.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-11T07:38:09.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:38:09.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:38:09.801 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:09 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:38:09.905 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:09 smithi164 ceph-mon[33167]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T07:38:09.905 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:09 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:09.905 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:09 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:09.905 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:09 smithi164 ceph-mon[33167]: Rotating authentication key for mgr.x 2024-09-11T07:38:09.905 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:09 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-11T07:38:09.905 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:09 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-11T07:38:09.905 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:09 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:38:09.905 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:09 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:38:09.905 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:09 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:38:10.217 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.071+0000 7f8bdd7cc640 -1 mgr.server reply reply (13) Permission denied access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication 2024-09-11T07:38:10.217 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.073+0000 7f8bdafc7640 -1 log_channel(cephadm) log [ERR] : Non-zero return from ['ceph', '-k', '/var/lib/ceph/mgr/ceph-x/keyring', '-n', 'mgr.x', 'tell', 'mgr.x', 'rotate-key', '-i', '-']: 2024-09-11T07:38:10.066+0000 7fd571ffb640 1 -- 172.21.15.164:0/3208076913 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7fd55c003710 con 0x7fd574066a50 2024-09-11T07:38:10.217 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.066+0000 7fd571ffb640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 13145685030724854681 != 18374858748799134293 2024-09-11T07:38:10.217 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.067+0000 7fd579641640 1 -- 172.21.15.164:0/3208076913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd574066a50 msgr2=0x7fd57410a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:38:10.217 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.067+0000 7fd579641640 1 --2- 172.21.15.164:0/3208076913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd574066a50 0x7fd57410a940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd568004950 tx=0x7fd568009da0 comp rx=0 tx=0).stop 2024-09-11T07:38:10.217 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.067+0000 7fd579641640 1 -- 172.21.15.164:0/3208076913 shutdown_connections 2024-09-11T07:38:10.217 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.067+0000 7fd579641640 1 --2- 172.21.15.164:0/3208076913 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd57410ae80 0x7fd57410d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T07:38:10.217 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.067+0000 7fd579641640 1 --2- 172.21.15.164:0/3208076913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd574066a50 0x7fd57410a940 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T07:38:10.217 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.067+0000 7fd579641640 1 --2- 172.21.15.164:0/3208076913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd574066100 0x7fd574066510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T07:38:10.217 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.067+0000 7fd579641640 1 -- 172.21.15.164:0/3208076913 >> 172.21.15.164:0/3208076913 conn(0x7fd57406a8d0 msgr2=0x7fd57406cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:38:10.217 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.067+0000 7fd579641640 1 -- 172.21.15.164:0/3208076913 shutdown_connections 2024-09-11T07:38:10.218 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.067+0000 7fd579641640 1 -- 172.21.15.164:0/3208076913 wait complete. 2024-09-11T07:38:10.218 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.067+0000 7fd579641640 1 Processor -- start 2024-09-11T07:38:10.218 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd579641640 1 -- start start 2024-09-11T07:38:10.218 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd579641640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd574066100 0x7fd574115550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T07:38:10.218 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd579641640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd574066a50 0x7fd574115a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T07:38:10.218 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd579641640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd57410ae80 0x7fd57411cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T07:38:10.218 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd579641640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57411d0b0 con 0x7fd574066100 2024-09-11T07:38:10.218 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd579641640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57411d220 con 0x7fd57410ae80 2024-09-11T07:38:10.218 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd579641640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57411d3e0 con 0x7fd574066a50 2024-09-11T07:38:10.218 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd5737fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd57410ae80 0x7fd57411cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T07:38:10.218 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd5737fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd57410ae80 0x7fd57411cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37314/0 (socket says 172.21.15.164:37314) 2024-09-11T07:38:10.218 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd5737fe640 1 -- 172.21.15.164:0/3020540364 learned_addr learned my addr 172.21.15.164:0/3020540364 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-11T07:38:10.218 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd572ffd640 1 --2- 172.21.15.164:0/3020540364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd574066100 0x7fd574115550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T07:38:10.218 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd5727fc640 1 --2- 172.21.15.164:0/3020540364 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd574066a50 0x7fd574115a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T07:38:10.219 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd5737fe640 1 -- 172.21.15.164:0/3020540364 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd574066a50 msgr2=0x7fd574115a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:38:10.219 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd5737fe640 1 --2- 172.21.15.164:0/3020540364 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd574066a50 0x7fd574115a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T07:38:10.219 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd5737fe640 1 -- 172.21.15.164:0/3020540364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd574066100 msgr2=0x7fd574115550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:38:10.219 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd5737fe640 1 --2- 172.21.15.164:0/3020540364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd574066100 0x7fd574115550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T07:38:10.219 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd5737fe640 1 -- 172.21.15.164:0/3020540364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd568005040 con 0x7fd57410ae80 2024-09-11T07:38:10.219 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd5737fe640 1 --2- 172.21.15.164:0/3020540364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd57410ae80 0x7fd57411cb70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00db80 tx=0x7fd56c004e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T07:38:10.219 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd55b7fe640 1 -- 172.21.15.164:0/3020540364 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c017070 con 0x7fd57410ae80 2024-09-11T07:38:10.219 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd55b7fe640 1 -- 172.21.15.164:0/3020540364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- auth(proto 2 2 bytes epoch 0) v1 -- 0x7fd55c0032d0 con 0x7fd57410ae80 2024-09-11T07:38:10.219 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd579641640 1 -- 172.21.15.164:0/3020540364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57411d6c0 con 0x7fd57410ae80 2024-09-11T07:38:10.219 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd55b7fe640 1 -- 172.21.15.164:0/3020540364 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(38 keys) v1 ==== 1752+0+0 (secure 0 0 0) 0x7fd56c00f040 con 0x7fd57410ae80 2024-09-11T07:38:10.219 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd55b7fe640 1 -- 172.21.15.164:0/3020540364 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c013720 con 0x7fd57410ae80 2024-09-11T07:38:10.219 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.068+0000 7fd579641640 1 -- 172.21.15.164:0/3020540364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57411dbe0 con 0x7fd57410ae80 2024-09-11T07:38:10.219 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.069+0000 7fd55b7fe640 1 -- 172.21.15.164:0/3020540364 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x7fd57411dbe0 con 0x7fd57410ae80 2024-09-11T07:38:10.219 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.069+0000 7fd55b7fe640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 13145685030724854681 != 18374858748799134293 2024-09-11T07:38:10.220 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.070+0000 7fd55b7fe640 1 -- 172.21.15.164:0/3020540364 <== mon.1 v2:172.21.15.164:3300/0 5 ==== mgrmap(e 26) v1 ==== 90804+0+0 (secure 0 0 0) 0x7fd56c023020 con 0x7fd57410ae80 2024-09-11T07:38:10.220 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.070+0000 7fd55b7fe640 1 --2- 172.21.15.164:0/3020540364 >> v2:172.21.15.164:6816/2945890484 conn(0x7fd55c06f1c0 0x7fd55c071670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T07:38:10.220 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.070+0000 7fd55b7fe640 1 -- 172.21.15.164:0/3020540364 --> v2:172.21.15.164:6816/2945890484 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd55c071d40 con 0x7fd55c06f1c0 2024-09-11T07:38:10.220 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.070+0000 7fd572ffd640 1 --2- 172.21.15.164:0/3020540364 >> v2:172.21.15.164:6816/2945890484 conn(0x7fd55c06f1c0 0x7fd55c071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T07:38:10.220 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.070+0000 7fd55b7fe640 1 -- 172.21.15.164:0/3020540364 <== mon.1 v2:172.21.15.164:3300/0 6 ==== osd_map(65..65 src has 1..65) v4 ==== 6499+0+0 (secure 0 0 0) 0x7fd56c08ef00 con 0x7fd57410ae80 2024-09-11T07:38:10.220 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.071+0000 7fd572ffd640 1 --2- 172.21.15.164:0/3020540364 >> v2:172.21.15.164:6816/2945890484 conn(0x7fd55c06f1c0 0x7fd55c071670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd574116ad0 tx=0x7fd56400b040 comp rx=0 tx=0).ready entity=mgr.24451 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T07:38:10.220 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.071+0000 7fd55b7fe640 1 -- 172.21.15.164:0/3020540364 <== mgr.24451 v2:172.21.15.164:6816/2945890484 1 ==== command_reply(tid 0: -13 access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication) v1 ==== 134+0+0 (secure 0 0 0) 0x7fd55c071d40 con 0x7fd55c06f1c0 2024-09-11T07:38:10.220 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: Error EACCES: problem getting command descriptions from mgr.x 2024-09-11T07:38:10.220 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.220 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.082+0000 7f8c003f3640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-11T07:38:10.220 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ignoring --setuser ceph since I am not root 2024-09-11T07:38:10.220 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: ignoring --setgroup ceph since I am not root 2024-09-11T07:38:10.220 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-mgr[35255]: -- 172.21.15.164:0/2164442101 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564682292340 con 0x56468229ec00 2024-09-11T07:38:10.220 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-mgr[35255]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 13145685030724854681 != 18374858748799134293 2024-09-11T07:38:10.220 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.214+0000 7fe0570e7ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:38:10.391 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:10.325+0000 7fa24a00dec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:38:10.391 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:10.389+0000 7fa24a00dec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:38:10.466 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.272+0000 7fe0570e7ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:38:10.466 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.336+0000 7fe0570e7ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:38:10.466 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.460+0000 7fe0570e7ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:38:10.798 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-mon[38451]: Reconfiguring daemon mgr.x on smithi164 2024-09-11T07:38:10.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:10.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:10.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-11T07:38:10.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-mon[38451]: Activating manager daemon y 2024-09-11T07:38:10.799 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-mon[38451]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T07:38:10.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-mon[38451]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-11T07:38:10.800 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-mon[38451]: mgrmap e27: y(active, starting, since 0.00626937s) 2024-09-11T07:38:10.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-mon[29604]: Reconfiguring daemon mgr.x on smithi164 2024-09-11T07:38:10.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:10.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-11T07:38:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-mon[29604]: Activating manager daemon y 2024-09-11T07:38:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-mon[29604]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T07:38:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-mon[29604]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-11T07:38:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:10 smithi143 ceph-mon[29604]: mgrmap e27: y(active, starting, since 0.00626937s) 2024-09-11T07:38:10.853 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.597+0000 7fe0570e7ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:38:10.854 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-mon[33167]: Reconfiguring daemon mgr.x on smithi164 2024-09-11T07:38:10.854 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:10.854 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' 2024-09-11T07:38:10.854 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-11T07:38:10.854 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-mon[33167]: Activating manager daemon y 2024-09-11T07:38:10.854 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-mon[33167]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T07:38:10.854 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-mon[33167]: from='mgr.24451 172.21.15.164:0/1371877543' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-11T07:38:10.854 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-mon[33167]: mgrmap e27: y(active, starting, since 0.00626937s) 2024-09-11T07:38:11.216 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.852+0000 7fe0570e7ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:38:11.216 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:10.911+0000 7fe0570e7ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:38:11.216 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:11.073+0000 7fe0570e7ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:38:11.394 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:11.100+0000 7fa24a00dec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:38:11.394 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:11.212+0000 7fa24a00dec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:38:11.394 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:11.332+0000 7fa24a00dec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:38:11.394 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:11.392+0000 7fa24a00dec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:38:11.777 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:11.518+0000 7fa24a00dec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:38:11.777 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:11.584+0000 7fa24a00dec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:38:11.777 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:11.641+0000 7fa24a00dec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:38:12.042 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:11.775+0000 7fa24a00dec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:38:12.043 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:11.838+0000 7fa24a00dec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:38:12.043 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:11.903+0000 7fa24a00dec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:38:12.043 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:11 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:11.983+0000 7fa24a00dec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:38:12.216 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:11.915+0000 7fe0570e7ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:38:12.216 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:11.981+0000 7fe0570e7ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:38:12.298 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:12.041+0000 7fa24a00dec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:38:12.298 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:12.097+0000 7fa24a00dec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:38:12.531 INFO:teuthology.orchestra.run.smithi143.stderr:++ ceph auth get-key mgr.x 2024-09-11T07:38:12.843 INFO:teuthology.orchestra.run.smithi143.stderr:+ NK=AQDgSOFm1RMPIhAAlz7pyUw+up4QBvtek8xxbg== 2024-09-11T07:38:12.843 INFO:teuthology.orchestra.run.smithi143.stderr:+ '[' AQApRuFmWI7XAxAAyTmYHRBvchxRRPeyvRwBCg== == AQDgSOFm1RMPIhAAlz7pyUw+up4QBvtek8xxbg== ']' 2024-09-11T07:38:12.966 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:12.683+0000 7fe0570e7ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:38:12.966 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:12.794+0000 7fe0570e7ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:38:12.967 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:12.918+0000 7fe0570e7ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:38:12.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='client.? 172.21.15.143:0/1606922112' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T07:38:12.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: Active manager daemon y restarted 2024-09-11T07:38:12.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: Activating manager daemon y 2024-09-11T07:38:12.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.? 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T07:38:12.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T07:38:12.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: mgrmap e28: y(active, starting, since 0.00691964s) 2024-09-11T07:38:12.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T07:38:12.967 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:38:12.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:38:12.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:38:12.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:38:12.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:38:12.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:38:12.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:38:12.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:38:12.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:38:12.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:38:12.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:38:12.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:38:12.968 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:38:12.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:38:12.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:38:12.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:38:12.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T07:38:12.969 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:38:13.192 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:12.856+0000 7fa24a00dec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:38:13.192 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: [11/Sep/2024:07:38:12] ENGINE Bus STARTING 2024-09-11T07:38:13.192 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: CherryPy Checker: 2024-09-11T07:38:13.192 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: The Application mounted at '' has an empty config. 2024-09-11T07:38:13.192 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:13.192 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: [11/Sep/2024:07:38:12] ENGINE Serving on http://:::9283 2024-09-11T07:38:13.192 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: [11/Sep/2024:07:38:12] ENGINE Bus STARTED 2024-09-11T07:38:13.193 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: [11/Sep/2024:07:38:12] ENGINE Bus STOPPING 2024-09-11T07:38:13.193 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: [11/Sep/2024:07:38:12] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:38:13.193 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: [11/Sep/2024:07:38:12] ENGINE Bus STOPPED 2024-09-11T07:38:13.193 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: [11/Sep/2024:07:38:13] ENGINE Bus STARTING 2024-09-11T07:38:13.193 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='client.? 172.21.15.143:0/1606922112' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T07:38:13.193 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: Active manager daemon y restarted 2024-09-11T07:38:13.193 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: Activating manager daemon y 2024-09-11T07:38:13.194 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.? 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T07:38:13.194 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T07:38:13.194 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: mgrmap e28: y(active, starting, since 0.00691964s) 2024-09-11T07:38:13.194 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T07:38:13.194 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:38:13.194 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:38:13.194 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:38:13.194 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:38:13.194 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:38:13.194 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:38:13.194 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:38:13.194 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:38:13.194 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:38:13.195 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:38:13.195 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:38:13.195 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:38:13.195 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:38:13.195 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:38:13.195 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:38:13.195 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:38:13.195 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T07:38:13.195 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:38:13.195 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='client.? 172.21.15.143:0/1606922112' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-11T07:38:13.196 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: Active manager daemon y restarted 2024-09-11T07:38:13.196 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: Activating manager daemon y 2024-09-11T07:38:13.196 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.? 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T07:38:13.196 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T07:38:13.196 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: mgrmap e28: y(active, starting, since 0.00691964s) 2024-09-11T07:38:13.196 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T07:38:13.196 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:38:13.196 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:38:13.196 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:38:13.196 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:38:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:38:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:38:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:38:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:38:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:38:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:38:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:38:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:38:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:38:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:38:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:38:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:38:13.198 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T07:38:13.198 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:12 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:38:13.235 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:12 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:12.978+0000 7fe0570e7ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:38:13.235 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:13.104+0000 7fe0570e7ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:38:13.235 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:13.173+0000 7fe0570e7ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:38:13.235 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:13.233+0000 7fe0570e7ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:38:13.513 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:13.373+0000 7fe0570e7ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:38:13.513 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:13.442+0000 7fe0570e7ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:38:13.513 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:13.511+0000 7fe0570e7ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:38:13.544 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: [11/Sep/2024:07:38:13] ENGINE Serving on http://:::9283 2024-09-11T07:38:13.544 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: [11/Sep/2024:07:38:13] ENGINE Bus STARTED 2024-09-11T07:38:13.570 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T07:38:13.580 INFO:tasks.cephadm:Teardown begin 2024-09-11T07:38:13.581 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:38:13.608 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:38:13.636 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T07:38:13.637 DEBUG:teuthology.orchestra.run.smithi143:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T07:38:13.664 DEBUG:teuthology.orchestra.run.smithi164:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T07:38:13.695 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T07:38:13.695 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T07:38:13.695 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mon.a 2024-09-11T07:38:13.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:13 smithi143 systemd[1]: Stopping Ceph mon.a for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:38:13.888 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:13.597+0000 7fe0570e7ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:38:13.888 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:13.661+0000 7fe0570e7ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:38:13.889 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:13.727+0000 7fe0570e7ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:38:14.182 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug there is no tcmu-runner data available 2024-09-11T07:38:14.183 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-mon[33167]: Manager daemon y is now available 2024-09-11T07:38:14.183 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:38:14.183 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:38:14.183 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:38:14.183 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-mon[33167]: from='mgr.25289 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:38:14.183 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-mon[33167]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:38:14.183 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:13 smithi164 ceph-mon[33167]: from='mgr.25289 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:38:14.197 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-mon[29604]: Manager daemon y is now available 2024-09-11T07:38:14.198 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:38:14.198 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:38:14.198 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:38:14.198 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-mon[29604]: from='mgr.25289 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:38:14.198 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-mon[29604]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:38:14.198 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-mon[29604]: from='mgr.25289 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:38:14.198 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-mon[38451]: Manager daemon y is now available 2024-09-11T07:38:14.198 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:38:14.198 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:38:14.198 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:38:14.198 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-mon[38451]: from='mgr.25289 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:38:14.198 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-mon[38451]: from='mgr.25289 172.21.15.143:0/533508594' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:38:14.199 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:13 smithi143 ceph-mon[38451]: from='mgr.25289 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:38:14.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:14.349+0000 7fa1ecb8e640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:07:38:14] ENGINE Error in HTTPServer.serve 2024-09-11T07:38:14.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: Traceback (most recent call last): 2024-09-11T07:38:14.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:38:14.549 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: self._connections.run(self.expiration_interval) 2024-09-11T07:38:14.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:38:14.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: self._run(expiration_interval) 2024-09-11T07:38:14.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:38:14.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:38:14.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:38:14.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:38:14.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:38:14.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: s = self.context.wrap_socket( 2024-09-11T07:38:14.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:38:14.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: return self.sslsocket_class._create( 2024-09-11T07:38:14.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:38:14.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: self.do_handshake() 2024-09-11T07:38:14.550 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:38:14.551 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: self._sslobj.do_handshake() 2024-09-11T07:38:14.551 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:38:14.551 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:14.936 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:14.526+0000 7fe0570e7ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:38:14.936 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: [11/Sep/2024:07:38:14] ENGINE Bus STARTING 2024-09-11T07:38:14.936 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: CherryPy Checker: 2024-09-11T07:38:14.936 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: The Application mounted at '' has an empty config. 2024-09-11T07:38:14.936 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: 2024-09-11T07:38:14.936 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: [11/Sep/2024:07:38:14] ENGINE Serving on http://:::9283 2024-09-11T07:38:14.936 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x[35232]: [11/Sep/2024:07:38:14] ENGINE Bus STARTED 2024-09-11T07:38:15.017 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-a[29581]: 2024-09-11T07:38:14.643+0000 7f9ffe14d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:38:15.017 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:14 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-a[29581]: 2024-09-11T07:38:14.643+0000 7f9ffe14d640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T07:38:15.017 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:14 smithi143 podman[83882]: 2024-09-11 07:38:14.71541319 +0000 UTC m=+0.190209851 container died d3f31892115fcf7c385237250325102518aa14580e3acbc8359b03863bb4e8f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:38:15.298 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:15 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: ::ffff:172.21.15.164 - - [11/Sep/2024:07:38:15] "GET /metrics HTTP/1.1" 200 34929 "" "Prometheus/2.43.0" 2024-09-11T07:38:15.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:15 smithi143 podman[83882]: 2024-09-11 07:38:15.116984326 +0000 UTC m=+0.591780986 container cleanup d3f31892115fcf7c385237250325102518aa14580e3acbc8359b03863bb4e8f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T07:38:15.299 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 11 07:38:15 smithi143 bash[83882]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-a 2024-09-11T07:38:15.515 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T07:38:15.564 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:38:15.564 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T07:38:15.565 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-11T07:38:15.565 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mon.c 2024-09-11T07:38:15.958 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:15 smithi143 systemd[1]: Stopping Ceph mon.c for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:38:16.772 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:16 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-c[38428]: 2024-09-11T07:38:16.437+0000 7f9e11143640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:38:16.772 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:16 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-c[38428]: 2024-09-11T07:38:16.437+0000 7f9e11143640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T07:38:16.772 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:16 smithi143 podman[84235]: 2024-09-11 07:38:16.489088863 +0000 UTC m=+0.413380060 container died 98bc9ad1527a5b88433c305102834ea725e2f695b36cd934c26ebbfae4b3eafd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T07:38:17.048 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:16 smithi143 podman[84235]: 2024-09-11 07:38:16.87819843 +0000 UTC m=+0.802489616 container cleanup 98bc9ad1527a5b88433c305102834ea725e2f695b36cd934c26ebbfae4b3eafd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T07:38:17.049 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:16 smithi143 bash[84235]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-c 2024-09-11T07:38:17.485 INFO:journalctl@ceph.mon.c.smithi143.stdout:Sep 11 07:38:17 smithi143 podman[84285]: 2024-09-11 07:38:17.113559171 +0000 UTC m=+0.625426601 container remove 98bc9ad1527a5b88433c305102834ea725e2f695b36cd934c26ebbfae4b3eafd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:38:17.800 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T07:38:17.838 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:38:17.838 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-11T07:38:17.838 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-11T07:38:17.838 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mon.b 2024-09-11T07:38:18.174 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:17 smithi164 systemd[1]: Stopping Ceph mon.b for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:38:18.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:18 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-b[33144]: 2024-09-11T07:38:18.172+0000 7f4fbfb94640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:38:18.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:18 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-b[33144]: 2024-09-11T07:38:18.172+0000 7f4fbfb94640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T07:38:18.466 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:18 smithi164 podman[73886]: 2024-09-11 07:38:18.387204168 +0000 UTC m=+0.358706013 container died 66c186b9d9f174a05e52a617d15042246eab3a11d1d5e4fdca04683f18d69f51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True) 2024-09-11T07:38:18.898 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:18 smithi164 podman[73886]: 2024-09-11 07:38:18.695227864 +0000 UTC m=+0.666729709 container cleanup 66c186b9d9f174a05e52a617d15042246eab3a11d1d5e4fdca04683f18d69f51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T07:38:18.898 INFO:journalctl@ceph.mon.b.smithi164.stdout:Sep 11 07:38:18 smithi164 bash[73886]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mon-b 2024-09-11T07:38:19.052 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T07:38:19.102 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:38:19.102 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-11T07:38:19.102 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T07:38:19.103 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mgr.y 2024-09-11T07:38:19.416 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:19 smithi143 systemd[1]: Stopping Ceph mgr.y for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:38:19.714 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:19 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y[83332]: 2024-09-11T07:38:19.413+0000 7fa245fa2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:38:19.714 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:19 smithi143 podman[84450]: 2024-09-11 07:38:19.445340116 +0000 UTC m=+0.146677325 container died 49cfafd26c01840087d25af36172d22994dd823dac336dd824685d0c9066c581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2) 2024-09-11T07:38:20.048 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:19 smithi143 podman[84450]: 2024-09-11 07:38:19.830061327 +0000 UTC m=+0.531398538 container cleanup 49cfafd26c01840087d25af36172d22994dd823dac336dd824685d0c9066c581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:38:20.049 INFO:journalctl@ceph.mgr.y.smithi143.stdout:Sep 11 07:38:19 smithi143 bash[84450]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-y 2024-09-11T07:38:20.209 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T07:38:20.244 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:38:20.245 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T07:38:20.245 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T07:38:20.245 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mgr.x 2024-09-11T07:38:20.531 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:20 smithi164 systemd[1]: Stopping Ceph mgr.x for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:38:20.794 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:20 smithi164 podman[74019]: 2024-09-11 07:38:20.548098105 +0000 UTC m=+0.113797776 container died 5b741da46660f56896d976b3c7181be804af67e0f87abfa2c200648b1a793183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-09-11T07:38:21.079 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:20 smithi164 podman[74019]: 2024-09-11 07:38:20.900069126 +0000 UTC m=+0.465768797 container cleanup 5b741da46660f56896d976b3c7181be804af67e0f87abfa2c200648b1a793183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:38:21.079 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:20 smithi164 bash[74019]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x 2024-09-11T07:38:21.333 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:21 smithi164 podman[74036]: 2024-09-11 07:38:21.078685766 +0000 UTC m=+0.524538205 container remove 5b741da46660f56896d976b3c7181be804af67e0f87abfa2c200648b1a793183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:38:21.333 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 11 07:38:21 smithi164 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-09-11T07:38:21.355 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T07:38:21.392 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:38:21.393 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T07:38:21.393 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T07:38:21.393 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.0 2024-09-11T07:38:21.687 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:38:21 smithi143 systemd[1]: Stopping Ceph osd.0 for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:38:22.048 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:38:21 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-0[42706]: 2024-09-11T07:38:21.685+0000 7f2730089640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:38:22.049 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:38:21 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-0[42706]: 2024-09-11T07:38:21.685+0000 7f2730089640 -1 osd.0 67 *** Got signal Terminated *** 2024-09-11T07:38:22.049 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:38:21 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-0[42706]: 2024-09-11T07:38:21.685+0000 7f2730089640 -1 osd.0 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:38:27.298 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:38:27 smithi143 podman[84585]: 2024-09-11 07:38:27.017411521 +0000 UTC m=+5.432675245 container died 7770e8545286407213bdb5e519b3b8b6fcd3dbcb2d6059bb5cb42ee926b07498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T07:38:27.654 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:38:27 smithi143 podman[84585]: 2024-09-11 07:38:27.384701927 +0000 UTC m=+5.799965643 container cleanup 7770e8545286407213bdb5e519b3b8b6fcd3dbcb2d6059bb5cb42ee926b07498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T07:38:27.654 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:38:27 smithi143 bash[84585]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-0 2024-09-11T07:38:27.996 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:38:27 smithi143 podman[84605]: 2024-09-11 07:38:27.6531765 +0000 UTC m=+0.630655944 container remove 7770e8545286407213bdb5e519b3b8b6fcd3dbcb2d6059bb5cb42ee926b07498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T07:38:28.299 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:38:27 smithi143 podman[84699]: 2024-09-11 07:38:27.995691717 +0000 UTC m=+0.101849157 container create 93ffdf7c5d182fdaf69ce2f8deee0b21cb0d0a41d00dbfeed65a7f0ff0c41bfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:38:28.299 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:38:28 smithi143 podman[84699]: 2024-09-11 07:38:27.909733329 +0000 UTC m=+0.015890696 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:38:28.299 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:38:28 smithi143 podman[84699]: 2024-09-11 07:38:28.189955646 +0000 UTC m=+0.296113001 container init 93ffdf7c5d182fdaf69ce2f8deee0b21cb0d0a41d00dbfeed65a7f0ff0c41bfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T07:38:28.299 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:38:28 smithi143 podman[84699]: 2024-09-11 07:38:28.195161891 +0000 UTC m=+0.301319259 container start 93ffdf7c5d182fdaf69ce2f8deee0b21cb0d0a41d00dbfeed65a7f0ff0c41bfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T07:38:28.711 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:38:28 smithi143 podman[84699]: 2024-09-11 07:38:28.303512372 +0000 UTC m=+0.409669741 container attach 93ffdf7c5d182fdaf69ce2f8deee0b21cb0d0a41d00dbfeed65a7f0ff0c41bfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-09-11T07:38:28.712 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 11 07:38:28 smithi143 podman[84699]: 2024-09-11 07:38:28.369359149 +0000 UTC m=+0.475516514 container died 93ffdf7c5d182fdaf69ce2f8deee0b21cb0d0a41d00dbfeed65a7f0ff0c41bfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T07:38:29.043 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T07:38:29.081 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:38:29.081 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T07:38:29.081 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T07:38:29.081 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.1 2024-09-11T07:38:29.298 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:38:29 smithi143 systemd[1]: Stopping Ceph osd.1 for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:38:29.798 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:38:29 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-1[48887]: 2024-09-11T07:38:29.397+0000 7f85062c2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:38:29.798 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:38:29 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-1[48887]: 2024-09-11T07:38:29.397+0000 7f85062c2640 -1 osd.1 67 *** Got signal Terminated *** 2024-09-11T07:38:29.798 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:38:29 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-1[48887]: 2024-09-11T07:38:29.397+0000 7f85062c2640 -1 osd.1 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:38:35.000 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:38:34 smithi143 podman[84827]: 2024-09-11 07:38:34.729992644 +0000 UTC m=+5.425664568 container died a866588b22087618ce3d7beb590688a39dac3041110eb71f340df2a324fa15dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T07:38:35.298 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:38:35 smithi143 podman[84827]: 2024-09-11 07:38:35.10589534 +0000 UTC m=+5.801567317 container cleanup a866588b22087618ce3d7beb590688a39dac3041110eb71f340df2a324fa15dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-1, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:38:35.298 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:38:35 smithi143 bash[84827]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-1 2024-09-11T07:38:35.719 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:38:35 smithi143 podman[84849]: 2024-09-11 07:38:35.399509438 +0000 UTC m=+0.668494721 container remove a866588b22087618ce3d7beb590688a39dac3041110eb71f340df2a324fa15dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T07:38:36.037 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:38:35 smithi143 podman[84941]: 2024-09-11 07:38:35.621400178 +0000 UTC m=+0.015827761 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:38:36.037 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:38:35 smithi143 podman[84941]: 2024-09-11 07:38:35.733664092 +0000 UTC m=+0.128091664 container create e695d8337815e400c536dc93abac6b3e825b411431c02b08a3446043c8a696bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T07:38:36.038 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:38:35 smithi143 podman[84941]: 2024-09-11 07:38:35.953390396 +0000 UTC m=+0.347817974 container init e695d8337815e400c536dc93abac6b3e825b411431c02b08a3446043c8a696bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T07:38:36.038 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:38:35 smithi143 podman[84941]: 2024-09-11 07:38:35.958114652 +0000 UTC m=+0.352542229 container start e695d8337815e400c536dc93abac6b3e825b411431c02b08a3446043c8a696bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:38:36.038 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:38:36 smithi143 podman[84941]: 2024-09-11 07:38:36.036484578 +0000 UTC m=+0.430912152 container attach e695d8337815e400c536dc93abac6b3e825b411431c02b08a3446043c8a696bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T07:38:36.299 INFO:journalctl@ceph.osd.1.smithi143.stdout:Sep 11 07:38:36 smithi143 podman[84941]: 2024-09-11 07:38:36.135001889 +0000 UTC m=+0.529429474 container died e695d8337815e400c536dc93abac6b3e825b411431c02b08a3446043c8a696bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-11T07:38:36.733 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T07:38:36.772 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:38:36.772 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T07:38:36.772 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T07:38:36.772 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.2 2024-09-11T07:38:37.048 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:38:36 smithi143 systemd[1]: Stopping Ceph osd.2 for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:38:37.548 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:38:37 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-2[55254]: 2024-09-11T07:38:37.094+0000 7fabb2dee640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:38:37.549 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:38:37 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-2[55254]: 2024-09-11T07:38:37.094+0000 7fabb2dee640 -1 osd.2 67 *** Got signal Terminated *** 2024-09-11T07:38:37.549 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:38:37 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-2[55254]: 2024-09-11T07:38:37.094+0000 7fabb2dee640 -1 osd.2 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:38:42.798 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:38:42 smithi143 podman[85069]: 2024-09-11 07:38:42.505914213 +0000 UTC m=+5.513866987 container died a11c3f3bf0fed3b2daff3dfa7e4dd66374e70d33f4e35a3b76ef20e2a41db2e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0) 2024-09-11T07:38:43.138 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:38:42 smithi143 podman[85069]: 2024-09-11 07:38:42.909929405 +0000 UTC m=+5.917882185 container cleanup a11c3f3bf0fed3b2daff3dfa7e4dd66374e70d33f4e35a3b76ef20e2a41db2e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T07:38:43.138 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:38:42 smithi143 bash[85069]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-2 2024-09-11T07:38:43.473 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:38:43 smithi143 podman[85089]: 2024-09-11 07:38:43.137064434 +0000 UTC m=+0.622677603 container remove a11c3f3bf0fed3b2daff3dfa7e4dd66374e70d33f4e35a3b76ef20e2a41db2e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T07:38:43.765 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:38:43 smithi143 podman[85183]: 2024-09-11 07:38:43.374717473 +0000 UTC m=+0.015754059 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:38:43.766 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:38:43 smithi143 podman[85183]: 2024-09-11 07:38:43.478810026 +0000 UTC m=+0.119846608 container create fe2ea77ae098d7d8e4571735331386d2c60bcd1507456ead5bcc77926e3e1acf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-11T07:38:43.766 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:38:43 smithi143 podman[85183]: 2024-09-11 07:38:43.665018912 +0000 UTC m=+0.306055499 container init fe2ea77ae098d7d8e4571735331386d2c60bcd1507456ead5bcc77926e3e1acf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:38:43.766 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:38:43 smithi143 podman[85183]: 2024-09-11 07:38:43.669938801 +0000 UTC m=+0.310975383 container start fe2ea77ae098d7d8e4571735331386d2c60bcd1507456ead5bcc77926e3e1acf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-11T07:38:43.766 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:38:43 smithi143 podman[85183]: 2024-09-11 07:38:43.764735689 +0000 UTC m=+0.405772275 container attach fe2ea77ae098d7d8e4571735331386d2c60bcd1507456ead5bcc77926e3e1acf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:38:44.048 INFO:journalctl@ceph.osd.2.smithi143.stdout:Sep 11 07:38:43 smithi143 podman[85183]: 2024-09-11 07:38:43.843406795 +0000 UTC m=+0.484443385 container died fe2ea77ae098d7d8e4571735331386d2c60bcd1507456ead5bcc77926e3e1acf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T07:38:44.419 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T07:38:44.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:38:44.458 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T07:38:44.458 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T07:38:44.459 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.3 2024-09-11T07:38:44.677 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:38:44 smithi143 systemd[1]: Stopping Ceph osd.3 for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:38:45.048 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:38:44 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-3[61301]: 2024-09-11T07:38:44.806+0000 7fe058691640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:38:45.048 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:38:44 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-3[61301]: 2024-09-11T07:38:44.806+0000 7fe058691640 -1 osd.3 67 *** Got signal Terminated *** 2024-09-11T07:38:45.048 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:38:44 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-3[61301]: 2024-09-11T07:38:44.806+0000 7fe058691640 -1 osd.3 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:38:48.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:38:48 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:38:48.116+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:38:49.134 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:48 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4[38495]: 2024-09-11T07:38:48.752+0000 7fe627ad5640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:23.512281+0000 front 2024-09-11T07:38:23.512302+0000 (oldest deadline 2024-09-11T07:38:48.212153+0000) 2024-09-11T07:38:49.402 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:38:49 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:38:49.132+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:38:49.798 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:38:49 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-3[61301]: 2024-09-11T07:38:49.374+0000 7fe055cab640 -1 osd.3 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:23.229014+0000 front 2024-09-11T07:38:23.228997+0000 (oldest deadline 2024-09-11T07:38:48.529043+0000) 2024-09-11T07:38:50.097 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:49 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4[38495]: 2024-09-11T07:38:49.798+0000 7fe627ad5640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:23.512281+0000 front 2024-09-11T07:38:23.512302+0000 (oldest deadline 2024-09-11T07:38:48.212153+0000) 2024-09-11T07:38:50.360 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:38:50 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:38:50.095+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:38:50.531 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:38:50 smithi143 podman[85310]: 2024-09-11 07:38:50.264551476 +0000 UTC m=+5.577060699 container died 58a39f8dcdff771afeaaa8b26168e61f0349869297fbb8d0d27a5e8a9122a85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-11T07:38:50.716 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:38:50 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:38:50.536+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:38:50.798 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:38:50 smithi143 podman[85310]: 2024-09-11 07:38:50.637448859 +0000 UTC m=+5.949958082 container cleanup 58a39f8dcdff771afeaaa8b26168e61f0349869297fbb8d0d27a5e8a9122a85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-3, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T07:38:50.799 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:38:50 smithi143 bash[85310]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-3 2024-09-11T07:38:51.118 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:50 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4[38495]: 2024-09-11T07:38:50.754+0000 7fe627ad5640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:23.512281+0000 front 2024-09-11T07:38:23.512302+0000 (oldest deadline 2024-09-11T07:38:48.212153+0000) 2024-09-11T07:38:51.257 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:38:50 smithi143 podman[85344]: 2024-09-11 07:38:50.947432266 +0000 UTC m=+0.679572784 container remove 58a39f8dcdff771afeaaa8b26168e61f0349869297fbb8d0d27a5e8a9122a85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-11T07:38:51.466 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:38:51 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:38:51.116+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:25.076400+0000 front 2024-09-11T07:38:25.076370+0000 (oldest deadline 2024-09-11T07:38:50.376313+0000) 2024-09-11T07:38:51.467 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:38:51 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:38:51.119+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:38:51.548 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:38:51 smithi143 podman[85442]: 2024-09-11 07:38:51.159416956 +0000 UTC m=+0.015637618 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:38:51.548 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:38:51 smithi143 podman[85442]: 2024-09-11 07:38:51.256596373 +0000 UTC m=+0.112817084 container create 8ee132e543da0f1db38466c3ab65e7d5dc7d309db65bafbc698c6a20947aee77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-11T07:38:51.549 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:38:51 smithi143 podman[85442]: 2024-09-11 07:38:51.49278919 +0000 UTC m=+0.349009852 container init 8ee132e543da0f1db38466c3ab65e7d5dc7d309db65bafbc698c6a20947aee77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T07:38:51.549 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:38:51 smithi143 podman[85442]: 2024-09-11 07:38:51.4978813 +0000 UTC m=+0.354101963 container start 8ee132e543da0f1db38466c3ab65e7d5dc7d309db65bafbc698c6a20947aee77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS) 2024-09-11T07:38:51.750 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:38:51 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:38:51.490+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:38:51.964 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:38:51 smithi143 podman[85442]: 2024-09-11 07:38:51.584362923 +0000 UTC m=+0.440583589 container attach 8ee132e543da0f1db38466c3ab65e7d5dc7d309db65bafbc698c6a20947aee77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-11T07:38:51.964 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 11 07:38:51 smithi143 podman[85442]: 2024-09-11 07:38:51.668697639 +0000 UTC m=+0.524918307 container died 8ee132e543da0f1db38466c3ab65e7d5dc7d309db65bafbc698c6a20947aee77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:38:52.071 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:51 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4[38495]: 2024-09-11T07:38:51.748+0000 7fe627ad5640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:23.512281+0000 front 2024-09-11T07:38:23.512302+0000 (oldest deadline 2024-09-11T07:38:48.212153+0000) 2024-09-11T07:38:52.251 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T07:38:52.292 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:38:52.292 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T07:38:52.292 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T07:38:52.292 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.4 2024-09-11T07:38:52.323 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:38:52 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:38:52.069+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:25.076400+0000 front 2024-09-11T07:38:25.076370+0000 (oldest deadline 2024-09-11T07:38:50.376313+0000) 2024-09-11T07:38:52.323 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:38:52 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:38:52.115+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:38:52.616 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:52 smithi164 systemd[1]: Stopping Ceph osd.4 for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:38:52.617 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:52 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4[38495]: 2024-09-11T07:38:52.614+0000 7fe62a4bb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:38:52.617 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:38:52 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:38:52.532+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:38:52.966 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:52 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4[38495]: 2024-09-11T07:38:52.614+0000 7fe62a4bb640 -1 osd.4 67 *** Got signal Terminated *** 2024-09-11T07:38:52.966 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:52 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4[38495]: 2024-09-11T07:38:52.614+0000 7fe62a4bb640 -1 osd.4 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:38:52.966 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:52 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4[38495]: 2024-09-11T07:38:52.704+0000 7fe627ad5640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:23.512281+0000 front 2024-09-11T07:38:23.512302+0000 (oldest deadline 2024-09-11T07:38:48.212153+0000) 2024-09-11T07:38:53.466 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:38:53 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:38:53.046+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:25.076400+0000 front 2024-09-11T07:38:25.076370+0000 (oldest deadline 2024-09-11T07:38:50.376313+0000) 2024-09-11T07:38:53.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:38:53 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:38:53.087+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:38:53.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:38:53 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:38:53.542+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:38:53.966 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:53 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4[38495]: 2024-09-11T07:38:53.745+0000 7fe627ad5640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:23.512281+0000 front 2024-09-11T07:38:23.512302+0000 (oldest deadline 2024-09-11T07:38:48.212153+0000) 2024-09-11T07:38:54.466 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:38:54 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:38:54.085+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:25.076400+0000 front 2024-09-11T07:38:25.076370+0000 (oldest deadline 2024-09-11T07:38:50.376313+0000) 2024-09-11T07:38:54.467 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:38:54 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:38:54.115+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:38:54.966 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:54 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4[38495]: 2024-09-11T07:38:54.777+0000 7fe627ad5640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:23.512281+0000 front 2024-09-11T07:38:23.512302+0000 (oldest deadline 2024-09-11T07:38:48.212153+0000) 2024-09-11T07:38:54.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:38:54 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:38:54.556+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:38:55.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:38:55 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:38:55.116+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:38:55.467 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:38:55 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:38:55.049+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:25.076400+0000 front 2024-09-11T07:38:25.076370+0000 (oldest deadline 2024-09-11T07:38:50.376313+0000) 2024-09-11T07:38:55.467 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:38:55 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:38:55.049+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:33.277217+0000 front 2024-09-11T07:38:33.277254+0000 (oldest deadline 2024-09-11T07:38:54.977241+0000) 2024-09-11T07:38:55.823 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:38:55 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:38:55.517+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:38:55.823 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:38:55 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:38:55.517+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:38:56.098 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:55 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4[38495]: 2024-09-11T07:38:55.821+0000 7fe627ad5640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:23.512281+0000 front 2024-09-11T07:38:23.512302+0000 (oldest deadline 2024-09-11T07:38:48.212153+0000) 2024-09-11T07:38:56.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:38:56 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:38:56.096+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:38:56.466 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:38:56 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:38:56.097+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:25.076400+0000 front 2024-09-11T07:38:25.076370+0000 (oldest deadline 2024-09-11T07:38:50.376313+0000) 2024-09-11T07:38:56.466 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:38:56 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:38:56.097+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:33.277217+0000 front 2024-09-11T07:38:33.277254+0000 (oldest deadline 2024-09-11T07:38:54.977241+0000) 2024-09-11T07:38:56.798 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:38:56 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:38:56.528+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:38:56.798 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:38:56 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:38:56.528+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:38:57.101 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:56 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4[38495]: 2024-09-11T07:38:56.795+0000 7fe627ad5640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:23.512281+0000 front 2024-09-11T07:38:23.512302+0000 (oldest deadline 2024-09-11T07:38:48.212153+0000) 2024-09-11T07:38:57.101 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:56 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4[38495]: 2024-09-11T07:38:56.795+0000 7fe627ad5640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:33.513174+0000 front 2024-09-11T07:38:33.513131+0000 (oldest deadline 2024-09-11T07:38:56.413079+0000) 2024-09-11T07:38:57.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:38:57 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:38:57.098+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:38:57.466 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:38:57 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:38:57.101+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:25.076400+0000 front 2024-09-11T07:38:25.076370+0000 (oldest deadline 2024-09-11T07:38:50.376313+0000) 2024-09-11T07:38:57.466 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:38:57 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:38:57.101+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:33.277217+0000 front 2024-09-11T07:38:33.277254+0000 (oldest deadline 2024-09-11T07:38:54.977241+0000) 2024-09-11T07:38:57.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:38:57 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:38:57.526+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:38:57.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:38:57 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:38:57.526+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:38:58.348 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:38:58 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:38:58.082+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:25.076400+0000 front 2024-09-11T07:38:25.076370+0000 (oldest deadline 2024-09-11T07:38:50.376313+0000) 2024-09-11T07:38:58.349 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:38:58 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:38:58.082+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:33.277217+0000 front 2024-09-11T07:38:33.277254+0000 (oldest deadline 2024-09-11T07:38:54.977241+0000) 2024-09-11T07:38:58.349 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:38:58 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:38:58.069+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:38:58.349 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:38:58 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:38:58.069+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.213871+0000 front 2024-09-11T07:38:31.213857+0000 (oldest deadline 2024-09-11T07:38:57.614074+0000) 2024-09-11T07:38:58.349 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:58 smithi164 podman[74175]: 2024-09-11 07:38:58.094631188 +0000 UTC m=+5.608732337 container stop 16bb5d09c9cc8eaa7e2189f1a9eecd4a2f5ce085e4bb925ed84feebfc40b15d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD) 2024-09-11T07:38:58.350 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:58 smithi164 podman[74175]: 2024-09-11 07:38:58.120233171 +0000 UTC m=+5.634334311 container died 16bb5d09c9cc8eaa7e2189f1a9eecd4a2f5ce085e4bb925ed84feebfc40b15d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T07:38:58.606 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:38:58 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:38:58.515+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:38:58.606 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:38:58 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:38:58.515+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:38:58.606 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:58 smithi164 podman[74175]: 2024-09-11 07:38:58.445807589 +0000 UTC m=+5.959908726 container cleanup 16bb5d09c9cc8eaa7e2189f1a9eecd4a2f5ce085e4bb925ed84feebfc40b15d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T07:38:58.607 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:58 smithi164 bash[74175]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4 2024-09-11T07:38:58.966 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:58 smithi164 podman[74196]: 2024-09-11 07:38:58.605387029 +0000 UTC m=+0.502730470 container remove 16bb5d09c9cc8eaa7e2189f1a9eecd4a2f5ce085e4bb925ed84feebfc40b15d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:38:59.318 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:38:59 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:38:59.072+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:25.076400+0000 front 2024-09-11T07:38:25.076370+0000 (oldest deadline 2024-09-11T07:38:50.376313+0000) 2024-09-11T07:38:59.318 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:38:59 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:38:59.072+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:33.277217+0000 front 2024-09-11T07:38:33.277254+0000 (oldest deadline 2024-09-11T07:38:54.977241+0000) 2024-09-11T07:38:59.318 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:38:59 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:38:59.056+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:38:59.319 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:38:59 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:38:59.056+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.213871+0000 front 2024-09-11T07:38:31.213857+0000 (oldest deadline 2024-09-11T07:38:57.614074+0000) 2024-09-11T07:38:59.319 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:59 smithi164 podman[74289]: 2024-09-11 07:38:58.904974061 +0000 UTC m=+0.016294586 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:38:59.319 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:59 smithi164 podman[74289]: 2024-09-11 07:38:59.023259981 +0000 UTC m=+0.134580536 container create 5141da0d95bf86aa4299fe37a4988e9ddb496dc71a217b7fdab179a08e110917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-11T07:38:59.319 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:59 smithi164 podman[74289]: 2024-09-11 07:38:59.234086425 +0000 UTC m=+0.345406935 container init 5141da0d95bf86aa4299fe37a4988e9ddb496dc71a217b7fdab179a08e110917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:38:59.319 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:59 smithi164 podman[74289]: 2024-09-11 07:38:59.238689137 +0000 UTC m=+0.350009662 container start 5141da0d95bf86aa4299fe37a4988e9ddb496dc71a217b7fdab179a08e110917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-11T07:38:59.319 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:59 smithi164 podman[74289]: 2024-09-11 07:38:59.317277596 +0000 UTC m=+0.428598135 container attach 5141da0d95bf86aa4299fe37a4988e9ddb496dc71a217b7fdab179a08e110917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T07:38:59.716 INFO:journalctl@ceph.osd.4.smithi164.stdout:Sep 11 07:38:59 smithi164 podman[74289]: 2024-09-11 07:38:59.418760006 +0000 UTC m=+0.530080530 container died 5141da0d95bf86aa4299fe37a4988e9ddb496dc71a217b7fdab179a08e110917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T07:38:59.717 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:38:59 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:38:59.535+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:38:59.717 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:38:59 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:38:59.535+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:00.056 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T07:39:00.136 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:39:00.136 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T07:39:00.136 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T07:39:00.137 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.5 2024-09-11T07:39:00.426 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:00.089+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:25.076400+0000 front 2024-09-11T07:38:25.076370+0000 (oldest deadline 2024-09-11T07:38:50.376313+0000) 2024-09-11T07:39:00.426 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:00.089+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:33.277217+0000 front 2024-09-11T07:38:33.277254+0000 (oldest deadline 2024-09-11T07:38:54.977241+0000) 2024-09-11T07:39:00.426 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:00 smithi164 systemd[1]: Stopping Ceph osd.5 for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:39:00.427 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:00.103+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:39:00.427 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:00.103+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.213871+0000 front 2024-09-11T07:38:31.213857+0000 (oldest deadline 2024-09-11T07:38:57.614074+0000) 2024-09-11T07:39:00.716 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:00.580+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:00.716 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:00.580+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:00.716 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:00.474+0000 7f5fb65b3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:39:00.717 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:00.474+0000 7f5fb65b3640 -1 osd.5 67 *** Got signal Terminated *** 2024-09-11T07:39:00.717 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:00 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:00.474+0000 7f5fb65b3640 -1 osd.5 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:39:01.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:01.056+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:39:01.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:01.056+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.213871+0000 front 2024-09-11T07:38:31.213857+0000 (oldest deadline 2024-09-11T07:38:57.614074+0000) 2024-09-11T07:39:01.467 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:01.082+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:25.076400+0000 front 2024-09-11T07:38:25.076370+0000 (oldest deadline 2024-09-11T07:38:50.376313+0000) 2024-09-11T07:39:01.467 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:01.082+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:33.277217+0000 front 2024-09-11T07:38:33.277254+0000 (oldest deadline 2024-09-11T07:38:54.977241+0000) 2024-09-11T07:39:01.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:01.559+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:01.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:01 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:01.559+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:02.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:02 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:02.022+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:39:02.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:02 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:02.022+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.213871+0000 front 2024-09-11T07:38:31.213857+0000 (oldest deadline 2024-09-11T07:38:57.614074+0000) 2024-09-11T07:39:02.466 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:02 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:02.105+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:25.076400+0000 front 2024-09-11T07:38:25.076370+0000 (oldest deadline 2024-09-11T07:38:50.376313+0000) 2024-09-11T07:39:02.466 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:02 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:02.105+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:33.277217+0000 front 2024-09-11T07:38:33.277254+0000 (oldest deadline 2024-09-11T07:38:54.977241+0000) 2024-09-11T07:39:02.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:02 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:02.554+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:02.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:02 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:02.554+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:03.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:03 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:03.067+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:39:03.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:03 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:03.067+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.213871+0000 front 2024-09-11T07:38:31.213857+0000 (oldest deadline 2024-09-11T07:38:57.614074+0000) 2024-09-11T07:39:03.467 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:03 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:03.112+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:25.076400+0000 front 2024-09-11T07:38:25.076370+0000 (oldest deadline 2024-09-11T07:38:50.376313+0000) 2024-09-11T07:39:03.467 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:03 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:03.112+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:33.277217+0000 front 2024-09-11T07:38:33.277254+0000 (oldest deadline 2024-09-11T07:38:54.977241+0000) 2024-09-11T07:39:03.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:03 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:03.537+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:03.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:03 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:03.537+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:04.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:04 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:04.099+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:39:04.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:04 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:04.099+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.213871+0000 front 2024-09-11T07:38:31.213857+0000 (oldest deadline 2024-09-11T07:38:57.614074+0000) 2024-09-11T07:39:04.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:04 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:04.099+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:39.815558+0000 front 2024-09-11T07:38:39.815447+0000 (oldest deadline 2024-09-11T07:39:03.315384+0000) 2024-09-11T07:39:04.467 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:04 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:04.144+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:25.076400+0000 front 2024-09-11T07:38:25.076370+0000 (oldest deadline 2024-09-11T07:38:50.376313+0000) 2024-09-11T07:39:04.467 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:04 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:04.144+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:33.277217+0000 front 2024-09-11T07:38:33.277254+0000 (oldest deadline 2024-09-11T07:38:54.977241+0000) 2024-09-11T07:39:04.467 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:04 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:04.144+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:41.279298+0000 front 2024-09-11T07:38:41.279311+0000 (oldest deadline 2024-09-11T07:39:03.579241+0000) 2024-09-11T07:39:04.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:04 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:04.576+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:04.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:04 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:04.576+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:05.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:05 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:05.132+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:39:05.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:05 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:05.132+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.213871+0000 front 2024-09-11T07:38:31.213857+0000 (oldest deadline 2024-09-11T07:38:57.614074+0000) 2024-09-11T07:39:05.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:05 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:05.132+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:39.815558+0000 front 2024-09-11T07:38:39.815447+0000 (oldest deadline 2024-09-11T07:39:03.315384+0000) 2024-09-11T07:39:05.467 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:05 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:05.142+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:25.076400+0000 front 2024-09-11T07:38:25.076370+0000 (oldest deadline 2024-09-11T07:38:50.376313+0000) 2024-09-11T07:39:05.467 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:05 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:05.142+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:33.277217+0000 front 2024-09-11T07:38:33.277254+0000 (oldest deadline 2024-09-11T07:38:54.977241+0000) 2024-09-11T07:39:05.467 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:05 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5[44727]: 2024-09-11T07:39:05.142+0000 7f5fb0bc7640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:41.279298+0000 front 2024-09-11T07:38:41.279311+0000 (oldest deadline 2024-09-11T07:39:03.579241+0000) 2024-09-11T07:39:05.966 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:05 smithi164 podman[74417]: 2024-09-11 07:39:05.804456844 +0000 UTC m=+5.436382458 container died 7c52c6b0a5d982825308043a5ecac2f99f6d4af35c19214993ce66b750ddf47c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T07:39:05.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:05 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:05.555+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:05.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:05 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:05.555+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:05.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:05 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:05.555+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:38.901774+0000 front 2024-09-11T07:38:38.901720+0000 (oldest deadline 2024-09-11T07:39:04.801622+0000) 2024-09-11T07:39:06.384 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:06 smithi164 podman[74417]: 2024-09-11 07:39:06.182218074 +0000 UTC m=+5.814143687 container cleanup 7c52c6b0a5d982825308043a5ecac2f99f6d4af35c19214993ce66b750ddf47c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T07:39:06.384 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:06 smithi164 bash[74417]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5 2024-09-11T07:39:06.385 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:06 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:06.090+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:39:06.385 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:06 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:06.090+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.213871+0000 front 2024-09-11T07:38:31.213857+0000 (oldest deadline 2024-09-11T07:38:57.614074+0000) 2024-09-11T07:39:06.385 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:06 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:06.090+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:39.815558+0000 front 2024-09-11T07:38:39.815447+0000 (oldest deadline 2024-09-11T07:39:03.315384+0000) 2024-09-11T07:39:06.716 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:06 smithi164 podman[74438]: 2024-09-11 07:39:06.383368959 +0000 UTC m=+0.576123104 container remove 7c52c6b0a5d982825308043a5ecac2f99f6d4af35c19214993ce66b750ddf47c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:39:06.716 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:06 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:06.604+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:06.717 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:06 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:06.604+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:06.717 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:06 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:06.604+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:38.901774+0000 front 2024-09-11T07:38:38.901720+0000 (oldest deadline 2024-09-11T07:39:04.801622+0000) 2024-09-11T07:39:07.054 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:06 smithi164 podman[74531]: 2024-09-11 07:39:06.626463188 +0000 UTC m=+0.015928460 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:39:07.054 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:06 smithi164 podman[74531]: 2024-09-11 07:39:06.734197142 +0000 UTC m=+0.123662403 container create 952f736bdf042ae10f0d76003eb80746a4d6843ec4c1d91d5815b4035fa01af5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-11T07:39:07.054 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:06 smithi164 podman[74531]: 2024-09-11 07:39:06.945103484 +0000 UTC m=+0.334568746 container init 952f736bdf042ae10f0d76003eb80746a4d6843ec4c1d91d5815b4035fa01af5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-11T07:39:07.054 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:06 smithi164 podman[74531]: 2024-09-11 07:39:06.950269521 +0000 UTC m=+0.339734785 container start 952f736bdf042ae10f0d76003eb80746a4d6843ec4c1d91d5815b4035fa01af5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T07:39:07.054 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:07 smithi164 podman[74531]: 2024-09-11 07:39:07.053330003 +0000 UTC m=+0.442795275 container attach 952f736bdf042ae10f0d76003eb80746a4d6843ec4c1d91d5815b4035fa01af5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:39:07.466 INFO:journalctl@ceph.osd.5.smithi164.stdout:Sep 11 07:39:07 smithi164 podman[74531]: 2024-09-11 07:39:07.124232585 +0000 UTC m=+0.513697849 container died 952f736bdf042ae10f0d76003eb80746a4d6843ec4c1d91d5815b4035fa01af5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T07:39:07.467 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:07 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:07.108+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:39:07.467 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:07 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:07.108+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.213871+0000 front 2024-09-11T07:38:31.213857+0000 (oldest deadline 2024-09-11T07:38:57.614074+0000) 2024-09-11T07:39:07.467 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:07 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:07.108+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:39.815558+0000 front 2024-09-11T07:38:39.815447+0000 (oldest deadline 2024-09-11T07:39:03.315384+0000) 2024-09-11T07:39:07.741 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:07 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:07.643+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:07.741 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:07 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:07.643+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:07.741 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:07 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:07.643+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:38.901774+0000 front 2024-09-11T07:38:38.901720+0000 (oldest deadline 2024-09-11T07:39:04.801622+0000) 2024-09-11T07:39:07.767 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T07:39:07.807 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:39:07.807 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T07:39:07.807 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T07:39:07.807 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.6 2024-09-11T07:39:08.112 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:07 smithi164 systemd[1]: Stopping Ceph osd.6 for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:39:08.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:08 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:08.111+0000 7f33c4aea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:39:08.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:08 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:08.111+0000 7f33c4aea640 -1 osd.6 67 *** Got signal Terminated *** 2024-09-11T07:39:08.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:08 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:08.111+0000 7f33c4aea640 -1 osd.6 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:39:08.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:08 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:08.147+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:39:08.467 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:08 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:08.147+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.213871+0000 front 2024-09-11T07:38:31.213857+0000 (oldest deadline 2024-09-11T07:38:57.614074+0000) 2024-09-11T07:39:08.467 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:08 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:08.147+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:39.815558+0000 front 2024-09-11T07:38:39.815447+0000 (oldest deadline 2024-09-11T07:39:03.315384+0000) 2024-09-11T07:39:08.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:08 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:08.611+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:08.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:08 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:08.611+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:08.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:08 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:08.611+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:38.901774+0000 front 2024-09-11T07:38:38.901720+0000 (oldest deadline 2024-09-11T07:39:04.801622+0000) 2024-09-11T07:39:09.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:09 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:09.172+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:39:09.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:09 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:09.173+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.213871+0000 front 2024-09-11T07:38:31.213857+0000 (oldest deadline 2024-09-11T07:38:57.614074+0000) 2024-09-11T07:39:09.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:09 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:09.173+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:39.815558+0000 front 2024-09-11T07:38:39.815447+0000 (oldest deadline 2024-09-11T07:39:03.315384+0000) 2024-09-11T07:39:09.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:09 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:09.570+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:09.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:09 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:09.570+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:09.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:09 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:09.570+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:38.901774+0000 front 2024-09-11T07:38:38.901720+0000 (oldest deadline 2024-09-11T07:39:04.801622+0000) 2024-09-11T07:39:10.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:10.189+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:39:10.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:10.189+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.213871+0000 front 2024-09-11T07:38:31.213857+0000 (oldest deadline 2024-09-11T07:38:57.614074+0000) 2024-09-11T07:39:10.467 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:10.189+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:39.815558+0000 front 2024-09-11T07:38:39.815447+0000 (oldest deadline 2024-09-11T07:39:03.315384+0000) 2024-09-11T07:39:10.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:10.544+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:10.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:10.544+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:10.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:10 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:10.544+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:38.901774+0000 front 2024-09-11T07:38:38.901720+0000 (oldest deadline 2024-09-11T07:39:04.801622+0000) 2024-09-11T07:39:11.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:11.143+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:39:11.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:11.143+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.213871+0000 front 2024-09-11T07:38:31.213857+0000 (oldest deadline 2024-09-11T07:38:57.614074+0000) 2024-09-11T07:39:11.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:11.143+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:39.815558+0000 front 2024-09-11T07:38:39.815447+0000 (oldest deadline 2024-09-11T07:39:03.315384+0000) 2024-09-11T07:39:11.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:11.143+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6815 osd.3 since back 2024-09-11T07:38:49.616974+0000 front 2024-09-11T07:38:49.617066+0000 (oldest deadline 2024-09-11T07:39:10.617121+0000) 2024-09-11T07:39:11.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:11.538+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:11.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:11.538+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:11.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:11.538+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:38.901774+0000 front 2024-09-11T07:38:38.901720+0000 (oldest deadline 2024-09-11T07:39:04.801622+0000) 2024-09-11T07:39:11.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:11 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:11.538+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6815 osd.3 since back 2024-09-11T07:38:44.802209+0000 front 2024-09-11T07:38:44.802215+0000 (oldest deadline 2024-09-11T07:39:10.702064+0000) 2024-09-11T07:39:12.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:12 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:12.144+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.312392+0000 front 2024-09-11T07:38:24.312410+0000 (oldest deadline 2024-09-11T07:38:47.212351+0000) 2024-09-11T07:39:12.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:12 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:12.144+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.213871+0000 front 2024-09-11T07:38:31.213857+0000 (oldest deadline 2024-09-11T07:38:57.614074+0000) 2024-09-11T07:39:12.466 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:12 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:12.144+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:39.815558+0000 front 2024-09-11T07:38:39.815447+0000 (oldest deadline 2024-09-11T07:39:03.315384+0000) 2024-09-11T07:39:12.467 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:12 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6[51095]: 2024-09-11T07:39:12.144+0000 7f33c0901640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.143:6815 osd.3 since back 2024-09-11T07:38:49.616974+0000 front 2024-09-11T07:38:49.617066+0000 (oldest deadline 2024-09-11T07:39:10.617121+0000) 2024-09-11T07:39:12.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:12 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:12.550+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:12.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:12 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:12.550+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:12.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:12 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:12.550+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:38.901774+0000 front 2024-09-11T07:38:38.901720+0000 (oldest deadline 2024-09-11T07:39:04.801622+0000) 2024-09-11T07:39:12.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:12 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:12.550+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6815 osd.3 since back 2024-09-11T07:38:44.802209+0000 front 2024-09-11T07:38:44.802215+0000 (oldest deadline 2024-09-11T07:39:10.702064+0000) 2024-09-11T07:39:13.716 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:13 smithi164 podman[74658]: 2024-09-11 07:39:13.439775854 +0000 UTC m=+5.408552741 container died d8c9b964f52eff40777b75c103d5c06f43378d95fc59e7043949b86412e68ac3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-11T07:39:13.717 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:13 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:13.563+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:13.717 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:13 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:13.563+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:13.717 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:13 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:13.563+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:38.901774+0000 front 2024-09-11T07:38:38.901720+0000 (oldest deadline 2024-09-11T07:39:04.801622+0000) 2024-09-11T07:39:13.717 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:13 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:13.563+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6815 osd.3 since back 2024-09-11T07:38:44.802209+0000 front 2024-09-11T07:38:44.802215+0000 (oldest deadline 2024-09-11T07:39:10.702064+0000) 2024-09-11T07:39:14.061 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:13 smithi164 podman[74658]: 2024-09-11 07:39:13.818011418 +0000 UTC m=+5.786788305 container cleanup d8c9b964f52eff40777b75c103d5c06f43378d95fc59e7043949b86412e68ac3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T07:39:14.061 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:13 smithi164 bash[74658]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6 2024-09-11T07:39:14.370 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:14 smithi164 podman[74681]: 2024-09-11 07:39:14.060716533 +0000 UTC m=+0.611091142 container remove d8c9b964f52eff40777b75c103d5c06f43378d95fc59e7043949b86412e68ac3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T07:39:14.673 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:14.527+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:14.673 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:14.527+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:14.673 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:14.527+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:38.901774+0000 front 2024-09-11T07:38:38.901720+0000 (oldest deadline 2024-09-11T07:39:04.801622+0000) 2024-09-11T07:39:14.673 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:14 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:14.527+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6815 osd.3 since back 2024-09-11T07:38:44.802209+0000 front 2024-09-11T07:38:44.802215+0000 (oldest deadline 2024-09-11T07:39:10.702064+0000) 2024-09-11T07:39:14.674 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:14 smithi164 podman[74777]: 2024-09-11 07:39:14.36984935 +0000 UTC m=+0.098583084 container create 361c256ef07a11b33f8f9b2d247a7e3d7a8ce14161c6bba514a6abbf4508d355 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T07:39:14.674 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:14 smithi164 podman[74777]: 2024-09-11 07:39:14.287545225 +0000 UTC m=+0.016278971 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:39:14.674 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:14 smithi164 podman[74777]: 2024-09-11 07:39:14.58925617 +0000 UTC m=+0.317989904 container init 361c256ef07a11b33f8f9b2d247a7e3d7a8ce14161c6bba514a6abbf4508d355 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-11T07:39:14.674 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:14 smithi164 podman[74777]: 2024-09-11 07:39:14.593816844 +0000 UTC m=+0.322550586 container start 361c256ef07a11b33f8f9b2d247a7e3d7a8ce14161c6bba514a6abbf4508d355 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD) 2024-09-11T07:39:14.674 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:14 smithi164 podman[74777]: 2024-09-11 07:39:14.672280239 +0000 UTC m=+0.401013984 container attach 361c256ef07a11b33f8f9b2d247a7e3d7a8ce14161c6bba514a6abbf4508d355 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:39:14.966 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:14 smithi164 podman[74777]: 2024-09-11 07:39:14.767200844 +0000 UTC m=+0.495934579 container died 361c256ef07a11b33f8f9b2d247a7e3d7a8ce14161c6bba514a6abbf4508d355 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:39:15.310 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 11 07:39:15 smithi164 podman[74820]: 2024-09-11 07:39:15.292308046 +0000 UTC m=+0.515823103 container remove 361c256ef07a11b33f8f9b2d247a7e3d7a8ce14161c6bba514a6abbf4508d355 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-11T07:39:15.316 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T07:39:15.355 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:39:15.355 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T07:39:15.356 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T07:39:15.356 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.7 2024-09-11T07:39:15.563 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:15 smithi164 systemd[1]: Stopping Ceph osd.7 for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:39:15.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:15.561+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:15.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:15.561+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:15.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:15.561+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:38.901774+0000 front 2024-09-11T07:38:38.901720+0000 (oldest deadline 2024-09-11T07:39:04.801622+0000) 2024-09-11T07:39:15.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:15.561+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6815 osd.3 since back 2024-09-11T07:38:44.802209+0000 front 2024-09-11T07:38:44.802215+0000 (oldest deadline 2024-09-11T07:39:10.702064+0000) 2024-09-11T07:39:15.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:15.688+0000 7ff53c2f1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:39:15.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:15.688+0000 7ff53c2f1640 -1 osd.7 67 *** Got signal Terminated *** 2024-09-11T07:39:15.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:15 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:15.688+0000 7ff53c2f1640 -1 osd.7 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:39:16.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:16.558+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:16.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:16.558+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:16.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:16.558+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:38.901774+0000 front 2024-09-11T07:38:38.901720+0000 (oldest deadline 2024-09-11T07:39:04.801622+0000) 2024-09-11T07:39:16.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:16 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:16.558+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6815 osd.3 since back 2024-09-11T07:38:44.802209+0000 front 2024-09-11T07:38:44.802215+0000 (oldest deadline 2024-09-11T07:39:10.702064+0000) 2024-09-11T07:39:17.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:17.540+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:17.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:17.540+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:17.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:17.540+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:38.901774+0000 front 2024-09-11T07:38:38.901720+0000 (oldest deadline 2024-09-11T07:39:04.801622+0000) 2024-09-11T07:39:17.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:17 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:17.540+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6815 osd.3 since back 2024-09-11T07:38:44.802209+0000 front 2024-09-11T07:38:44.802215+0000 (oldest deadline 2024-09-11T07:39:10.702064+0000) 2024-09-11T07:39:18.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:18 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:18.579+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:18.966 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:18 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:18.579+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:18.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:18 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:18.579+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:38.901774+0000 front 2024-09-11T07:38:38.901720+0000 (oldest deadline 2024-09-11T07:39:04.801622+0000) 2024-09-11T07:39:18.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:18 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:18.579+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6815 osd.3 since back 2024-09-11T07:38:44.802209+0000 front 2024-09-11T07:38:44.802215+0000 (oldest deadline 2024-09-11T07:39:10.702064+0000) 2024-09-11T07:39:18.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:18 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:18.579+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.164:6802 osd.4 since back 2024-09-11T07:38:54.802802+0000 front 2024-09-11T07:38:54.802781+0000 (oldest deadline 2024-09-11T07:39:18.302765+0000) 2024-09-11T07:39:19.867 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:19 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:19.605+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:19.868 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:19 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:19.605+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:19.868 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:19 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:19.605+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:38.901774+0000 front 2024-09-11T07:38:38.901720+0000 (oldest deadline 2024-09-11T07:39:04.801622+0000) 2024-09-11T07:39:19.868 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:19 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:19.605+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6815 osd.3 since back 2024-09-11T07:38:44.802209+0000 front 2024-09-11T07:38:44.802215+0000 (oldest deadline 2024-09-11T07:39:10.702064+0000) 2024-09-11T07:39:19.868 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:19 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:19.605+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.164:6802 osd.4 since back 2024-09-11T07:38:54.802802+0000 front 2024-09-11T07:38:54.802781+0000 (oldest deadline 2024-09-11T07:39:18.302765+0000) 2024-09-11T07:39:20.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:20.576+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6803 osd.0 since back 2024-09-11T07:38:24.299968+0000 front 2024-09-11T07:38:24.299869+0000 (oldest deadline 2024-09-11T07:38:49.599878+0000) 2024-09-11T07:39:20.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:20.576+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6807 osd.1 since back 2024-09-11T07:38:31.300797+0000 front 2024-09-11T07:38:31.300933+0000 (oldest deadline 2024-09-11T07:38:54.800737+0000) 2024-09-11T07:39:20.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:20.576+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6811 osd.2 since back 2024-09-11T07:38:38.901774+0000 front 2024-09-11T07:38:38.901720+0000 (oldest deadline 2024-09-11T07:39:04.801622+0000) 2024-09-11T07:39:20.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:20.576+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.143:6815 osd.3 since back 2024-09-11T07:38:44.802209+0000 front 2024-09-11T07:38:44.802215+0000 (oldest deadline 2024-09-11T07:39:10.702064+0000) 2024-09-11T07:39:20.967 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:20 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7[57493]: 2024-09-11T07:39:20.576+0000 7ff53a10c640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.164:6802 osd.4 since back 2024-09-11T07:38:54.802802+0000 front 2024-09-11T07:38:54.802781+0000 (oldest deadline 2024-09-11T07:39:18.302765+0000) 2024-09-11T07:39:21.466 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:21 smithi164 podman[74905]: 2024-09-11 07:39:21.234054846 +0000 UTC m=+5.645689164 container died da7d247448503a5976e93de91d04034474ba131c02c5ed82702485d71674185a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T07:39:21.872 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:21 smithi164 podman[74905]: 2024-09-11 07:39:21.586785936 +0000 UTC m=+5.998420256 container cleanup da7d247448503a5976e93de91d04034474ba131c02c5ed82702485d71674185a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:39:21.872 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:21 smithi164 bash[74905]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7 2024-09-11T07:39:22.189 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:21 smithi164 podman[74937]: 2024-09-11 07:39:21.871109217 +0000 UTC m=+0.632139397 container remove da7d247448503a5976e93de91d04034474ba131c02c5ed82702485d71674185a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-11T07:39:22.466 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:22 smithi164 podman[75032]: 2024-09-11 07:39:22.188646404 +0000 UTC m=+0.104066535 container create 1571b7bcd74042cf17c303240bd4094ca4a16861fc0e9cb2f74e0bf7792b1e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:39:22.467 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:22 smithi164 podman[75032]: 2024-09-11 07:39:22.1003052 +0000 UTC m=+0.015725339 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:39:22.467 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:22 smithi164 podman[75032]: 2024-09-11 07:39:22.399537834 +0000 UTC m=+0.314957963 container init 1571b7bcd74042cf17c303240bd4094ca4a16861fc0e9cb2f74e0bf7792b1e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:39:22.467 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:22 smithi164 podman[75032]: 2024-09-11 07:39:22.404185092 +0000 UTC m=+0.319605232 container start 1571b7bcd74042cf17c303240bd4094ca4a16861fc0e9cb2f74e0bf7792b1e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T07:39:22.869 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:22 smithi164 podman[75032]: 2024-09-11 07:39:22.490764741 +0000 UTC m=+0.406184894 container attach 1571b7bcd74042cf17c303240bd4094ca4a16861fc0e9cb2f74e0bf7792b1e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-11T07:39:22.870 INFO:journalctl@ceph.osd.7.smithi164.stdout:Sep 11 07:39:22 smithi164 podman[75032]: 2024-09-11 07:39:22.580340053 +0000 UTC m=+0.495760196 container died 1571b7bcd74042cf17c303240bd4094ca4a16861fc0e9cb2f74e0bf7792b1e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-11T07:39:23.179 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T07:39:23.218 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:39:23.219 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T07:39:23.219 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-11T07:39:23.219 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-d3dfd926-700e-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-11T07:39:23.515 INFO:journalctl@ceph.rgw.foo.a.smithi143.stdout:Sep 11 07:39:23 smithi143 systemd[1]: Stopping Ceph rgw.foo.a for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:39:23.798 INFO:journalctl@ceph.rgw.foo.a.smithi143.stdout:Sep 11 07:39:23 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-rgw-foo-a[66256]: 2024-09-11T07:39:23.513+0000 7fac22db9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:39:23.799 INFO:journalctl@ceph.rgw.foo.a.smithi143.stdout:Sep 11 07:39:23 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-rgw-foo-a[66256]: 2024-09-11T07:39:23.513+0000 7fac272c17c0 -1 shutting down 2024-09-11T07:39:33.799 INFO:journalctl@ceph.rgw.foo.a.smithi143.stdout:Sep 11 07:39:33 smithi143 bash[85573]: time="2024-09-11T07:39:33Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-d3dfd926-700e-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-11T07:39:33.799 INFO:journalctl@ceph.rgw.foo.a.smithi143.stdout:Sep 11 07:39:33 smithi143 podman[85573]: 2024-09-11 07:39:33.54019905 +0000 UTC m=+10.133654828 container died 3a2c5787d564e48f122d9d9a1abe732696353eb22cc54e5c0df2c31442837e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T07:39:34.180 INFO:journalctl@ceph.rgw.foo.a.smithi143.stdout:Sep 11 07:39:33 smithi143 podman[85573]: 2024-09-11 07:39:33.960602785 +0000 UTC m=+10.554058573 container cleanup 3a2c5787d564e48f122d9d9a1abe732696353eb22cc54e5c0df2c31442837e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2) 2024-09-11T07:39:34.180 INFO:journalctl@ceph.rgw.foo.a.smithi143.stdout:Sep 11 07:39:33 smithi143 bash[85573]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-11T07:39:34.390 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-11T07:39:34.426 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:39:34.426 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-11T07:39:34.426 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-11T07:39:34.426 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-d3dfd926-700e-11ef-bcea-c7b262605968@prometheus.a 2024-09-11T07:39:34.716 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:39:34 smithi164 systemd[1]: Stopping Ceph prometheus.a for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:39:35.022 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:39:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:39:34.748Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T07:39:35.022 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:39:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:39:34.748Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T07:39:35.023 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:39:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:39:34.748Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T07:39:35.023 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:39:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:39:34.748Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T07:39:35.023 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:39:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:39:34.748Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T07:39:35.023 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:39:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:39:34.748Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T07:39:35.023 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:39:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:39:34.748Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T07:39:35.023 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:39:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:39:34.748Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T07:39:35.023 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:39:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:39:34.748Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T07:39:35.023 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:39:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:39:34.748Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T07:39:35.023 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:39:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:39:34.748Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T07:39:35.023 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:39:34 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a[68299]: ts=2024-09-11T07:39:34.748Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T07:39:35.023 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:39:34 smithi164 podman[75160]: 2024-09-11 07:39:34.758550058 +0000 UTC m=+0.136811972 container died 966edf34c783a662dc5aaac963f9063ec79f720b10e0f94b6128b33d788c85e1 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:39:35.372 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:39:35 smithi164 podman[75160]: 2024-09-11 07:39:35.129027786 +0000 UTC m=+0.507289690 container cleanup 966edf34c783a662dc5aaac963f9063ec79f720b10e0f94b6128b33d788c85e1 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:39:35.373 INFO:journalctl@ceph.prometheus.a.smithi164.stdout:Sep 11 07:39:35 smithi164 bash[75160]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T07:39:35.504 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d3dfd926-700e-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-11T07:39:35.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:39:35.539 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-11T07:39:35.540 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm rm-cluster --fsid d3dfd926-700e-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T07:39:37.548 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:39:37 smithi143 systemd[1]: Stopping Ceph alertmanager.a for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:39:37.885 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:39:37 smithi143 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a[76038]: ts=2024-09-11T07:39:37.560Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T07:39:37.885 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:39:37 smithi143 podman[86043]: 2024-09-11 07:39:37.591125664 +0000 UTC m=+0.213749934 container died 6d2fef2e434be6ab36c0ac3c441e70dfd4ff1a3fe9c8dfba2f374577d8a6a919 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:39:38.218 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:39:37 smithi143 podman[86043]: 2024-09-11 07:39:37.991328995 +0000 UTC m=+0.613953252 container cleanup 6d2fef2e434be6ab36c0ac3c441e70dfd4ff1a3fe9c8dfba2f374577d8a6a919 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:39:38.219 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:39:37 smithi143 bash[86043]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T07:39:38.479 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:39:38 smithi143 podman[86060]: 2024-09-11 07:39:38.217610307 +0000 UTC m=+0.616667754 container remove 6d2fef2e434be6ab36c0ac3c441e70dfd4ff1a3fe9c8dfba2f374577d8a6a919 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:39:38.480 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:39:38 smithi143 podman[86060]: 2024-09-11 07:39:38.301755975 +0000 UTC m=+0.700813421 volume remove 6574e6fc7bb75d7f1742a279cdd1141ab53feb07b3443e6c00dd3591a30b4412 2024-09-11T07:39:38.480 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:39:38 smithi143 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T07:39:38.480 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:39:38 smithi143 systemd[1]: Stopped Ceph alertmanager.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:39:38.480 INFO:journalctl@ceph.alertmanager.a.smithi143.stdout:Sep 11 07:39:38 smithi143 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@alertmanager.a.service: Consumed 1.246s CPU time. 2024-09-11T07:39:38.798 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:39:38 smithi143 systemd[1]: Stopping Ceph node-exporter.a for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:39:39.172 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:39:38 smithi143 podman[86174]: 2024-09-11 07:39:38.89270214 +0000 UTC m=+0.103598554 container died a75708e36fc0873f1d7d039364f2fc4d8ab310a9ce33c7642f9068fa589fcaf6 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:39:39.498 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:39:39 smithi143 podman[86174]: 2024-09-11 07:39:39.270900895 +0000 UTC m=+0.481797312 container cleanup a75708e36fc0873f1d7d039364f2fc4d8ab310a9ce33c7642f9068fa589fcaf6 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:39:39.498 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:39:39 smithi143 bash[86174]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a 2024-09-11T07:39:39.774 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:39:39 smithi143 podman[86191]: 2024-09-11 07:39:39.497030623 +0000 UTC m=+0.608667496 container remove a75708e36fc0873f1d7d039364f2fc4d8ab310a9ce33c7642f9068fa589fcaf6 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:39:39.774 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:39:39 smithi143 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T07:39:39.774 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:39:39 smithi143 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-11T07:39:39.775 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:39:39 smithi143 systemd[1]: Stopped Ceph node-exporter.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:39:39.775 INFO:journalctl@ceph.node-exporter.a.smithi143.stdout:Sep 11 07:39:39 smithi143 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 2.562s CPU time. 2024-09-11T07:39:40.246 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm rm-cluster --fsid d3dfd926-700e-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T07:39:41.966 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:41 smithi164 systemd[1]: Stopping Ceph iscsi.iscsi.a for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:39:41.966 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:41 smithi164 ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[62955]: debug Shutdown received 2024-09-11T07:39:52.083 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:51 smithi164 bash[75543]: time="2024-09-11T07:39:51Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-11T07:39:52.083 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:51 smithi164 podman[75543]: 2024-09-11 07:39:51.836501542 +0000 UTC m=+10.119495938 container died 10287ccb5bb223eab77b148be2f3d11d450f7cc6c1dbb343176cc9350a601218 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-11T07:39:52.361 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:52 smithi164 podman[75543]: 2024-09-11 07:39:52.198437756 +0000 UTC m=+10.481432143 container cleanup 10287ccb5bb223eab77b148be2f3d11d450f7cc6c1dbb343176cc9350a601218 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-09-11T07:39:52.362 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:52 smithi164 bash[75543]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-11T07:39:52.716 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:52 smithi164 podman[75568]: 2024-09-11 07:39:52.399821294 +0000 UTC m=+0.559841627 container remove 10287ccb5bb223eab77b148be2f3d11d450f7cc6c1dbb343176cc9350a601218 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:39:52.716 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:52 smithi164 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-11T07:39:53.069 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:52 smithi164 podman[75625]: 2024-09-11 07:39:52.773200869 +0000 UTC m=+0.127986815 container died 1b39f1f84043cd08734ea4cf10a5a84340f68089760210f8a8f16304ecdc169d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD) 2024-09-11T07:39:53.070 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:53 smithi164 podman[75625]: 2024-09-11 07:39:53.068850863 +0000 UTC m=+0.423636807 container cleanup 1b39f1f84043cd08734ea4cf10a5a84340f68089760210f8a8f16304ecdc169d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T07:39:53.465 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:53 smithi164 bash[75625]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-11T07:39:53.465 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:53 smithi164 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-11T07:39:53.465 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:53 smithi164 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 62809 (conmon) remains running after unit stopped. 2024-09-11T07:39:53.465 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:53 smithi164 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 75643 (podman) remains running after unit stopped. 2024-09-11T07:39:53.465 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:53 smithi164 systemd[1]: Stopped Ceph iscsi.iscsi.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:39:53.465 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:53 smithi164 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 2.707s CPU time. 2024-09-11T07:39:53.465 INFO:journalctl@ceph.iscsi.iscsi.a.smithi164.stdout:Sep 11 07:39:53 smithi164 podman[75643]: 2024-09-11 07:39:53.211764596 +0000 UTC m=+0.441920036 container remove 1b39f1f84043cd08734ea4cf10a5a84340f68089760210f8a8f16304ecdc169d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:39:53.716 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:39:53 smithi164 systemd[1]: Stopping Ceph grafana.a for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:39:53.716 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:39:53 smithi164 podman[75767]: 2024-09-11 07:39:53.70287093 +0000 UTC m=+0.100155691 container died c3571985e0e2784fd0d9f321b39df2237126af1082d6305fe68a8c1653efb194 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a, com.redhat.component=ubi8-container, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, description=Ceph Grafana Container, version=8.5, build-date=2022-04-29T11:48:11.055897, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, architecture=x86_64, release=239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.buildah.version=1.32.0, io.openshift.expose-services=, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, name=ubi8) 2024-09-11T07:39:54.342 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:39:54 smithi164 podman[75767]: 2024-09-11 07:39:54.056281004 +0000 UTC m=+0.453565767 container cleanup c3571985e0e2784fd0d9f321b39df2237126af1082d6305fe68a8c1653efb194 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a, io.buildah.version=1.32.0, summary=Grafana Container configured for Ceph mgr/dashboard integration, vendor=Red Hat, Inc., version=8.5, description=Ceph Grafana Container, com.redhat.component=ubi8-container, io.k8s.display-name=Red Hat Universal Base Image 8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, release=239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, name=ubi8, io.openshift.tags=base rhel8, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, io.openshift.expose-services=) 2024-09-11T07:39:54.343 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:39:54 smithi164 bash[75767]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a 2024-09-11T07:39:54.659 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:39:54 smithi164 podman[75784]: 2024-09-11 07:39:54.340992422 +0000 UTC m=+0.635612913 container remove c3571985e0e2784fd0d9f321b39df2237126af1082d6305fe68a8c1653efb194 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-grafana-a, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0, maintainer=Paul Cuzner , architecture=x86_64, io.openshift.tags=base rhel8, distribution-scope=public, com.redhat.component=ubi8-container, description=Ceph Grafana Container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vendor=Red Hat, Inc., name=ubi8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, release=239.1651231664, build-date=2022-04-29T11:48:11.055897, vcs-type=git, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56) 2024-09-11T07:39:54.660 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:39:54 smithi164 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T07:39:54.660 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:39:54 smithi164 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-11T07:39:54.660 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:39:54 smithi164 systemd[1]: Stopped Ceph grafana.a for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:39:54.660 INFO:journalctl@ceph.grafana.a.smithi164.stdout:Sep 11 07:39:54 smithi164 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@grafana.a.service: Consumed 6.993s CPU time. 2024-09-11T07:39:54.949 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:39:54 smithi164 systemd[1]: Stopping Ceph node-exporter.b for d3dfd926-700e-11ef-bcea-c7b262605968... 2024-09-11T07:39:55.216 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:39:54 smithi164 podman[75899]: 2024-09-11 07:39:54.948146407 +0000 UTC m=+0.148349128 container died b4feb6ba9bf94d8dc8696e5aefbb06605d5b5cb0671b110e9674fb159728d69a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:39:55.662 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:39:55 smithi164 podman[75899]: 2024-09-11 07:39:55.26877054 +0000 UTC m=+0.468973252 container cleanup b4feb6ba9bf94d8dc8696e5aefbb06605d5b5cb0671b110e9674fb159728d69a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:39:55.662 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:39:55 smithi164 bash[75899]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b 2024-09-11T07:39:55.662 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:39:55 smithi164 podman[75917]: 2024-09-11 07:39:55.453098714 +0000 UTC m=+0.508692506 container remove b4feb6ba9bf94d8dc8696e5aefbb06605d5b5cb0671b110e9674fb159728d69a (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d3dfd926-700e-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:39:55.662 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:39:55 smithi164 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-11T07:39:55.966 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:39:55 smithi164 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-11T07:39:55.966 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:39:55 smithi164 systemd[1]: Stopped Ceph node-exporter.b for d3dfd926-700e-11ef-bcea-c7b262605968. 2024-09-11T07:39:55.966 INFO:journalctl@ceph.node-exporter.b.smithi164.stdout:Sep 11 07:39:55 smithi164 systemd[1]: ceph-d3dfd926-700e-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 2.602s CPU time. 2024-09-11T07:39:56.322 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:39:56.355 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:39:56.382 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T07:39:56.384 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901214/remote/smithi143/crash 2024-09-11T07:39:56.385 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/crash -- . 2024-09-11T07:39:56.437 INFO:teuthology.orchestra.run.smithi143.stderr:tar: /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T07:39:56.437 INFO:teuthology.orchestra.run.smithi143.stderr:tar: Error is not recoverable: exiting now 2024-09-11T07:39:56.439 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901214/remote/smithi164/crash 2024-09-11T07:39:56.440 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/crash -- . 2024-09-11T07:39:56.472 INFO:teuthology.orchestra.run.smithi164.stderr:tar: /var/lib/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T07:39:56.473 INFO:teuthology.orchestra.run.smithi164.stderr:tar: Error is not recoverable: exiting now 2024-09-11T07:39:56.474 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T07:39:56.474 DEBUG:teuthology.orchestra.run.smithi143:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-11T07:39:56.506 INFO:teuthology.orchestra.run.smithi143.stderr:grep: /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-11T07:39:56.508 INFO:tasks.cephadm:Compressing logs... 2024-09-11T07:39:56.508 DEBUG:teuthology.orchestra.run.smithi143:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T07:39:56.552 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T07:39:56.577 INFO:teuthology.orchestra.run.smithi164.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-11T07:39:56.577 INFO:teuthology.orchestra.run.smithi164.stderr:: No such file or directory 2024-09-11T07:39:56.577 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T07:39:56.577 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T07:39:56.577 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/tcmu-runner.log: 63.1% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-11T07:39:56.577 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T07:39:56.578 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T07:39:56.578 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T07:39:56.578 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T07:39:56.579 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T07:39:56.579 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T07:39:56.579 INFO:teuthology.orchestra.run.smithi143.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T07:39:56.579 INFO:teuthology.orchestra.run.smithi143.stderr:: No such file or directory 2024-09-11T07:39:56.579 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T07:39:56.580 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.0.log: -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T07:39:56.580 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T07:39:56.581 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T07:39:56.582 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T07:39:56.582 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T07:39:56.582 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T07:39:56.583 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-11T07:39:56.587 INFO:teuthology.orchestra.run.smithi164.stderr: 91.1% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T07:39:56.589 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-mgr.y.log: 91.1% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T07:39:56.591 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T07:39:56.591 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: 58.5% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-11T07:39:56.591 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T07:39:56.604 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T07:39:56.613 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 91.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T07:39:56.735 INFO:teuthology.orchestra.run.smithi164.stderr: 92.2% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T07:39:56.785 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T07:39:56.801 INFO:teuthology.orchestra.run.smithi143.stderr: 92.2% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T07:39:56.876 INFO:teuthology.orchestra.run.smithi164.stderr: 93.0% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T07:39:56.916 INFO:teuthology.orchestra.run.smithi164.stderr: 93.2% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T07:39:56.958 INFO:teuthology.orchestra.run.smithi164.stderr: 93.0% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T07:39:56.989 INFO:teuthology.orchestra.run.smithi143.stderr: 93.0% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T07:39:56.993 INFO:teuthology.orchestra.run.smithi143.stderr: 92.7% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T07:39:56.995 INFO:teuthology.orchestra.run.smithi164.stderr: 93.2% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T07:39:56.997 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-09-11T07:39:56.997 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m0.431s 2024-09-11T07:39:56.997 INFO:teuthology.orchestra.run.smithi164.stderr:user 0m1.577s 2024-09-11T07:39:56.997 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m0.080s 2024-09-11T07:39:57.122 INFO:teuthology.orchestra.run.smithi143.stderr: 93.3% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T07:39:57.128 INFO:teuthology.orchestra.run.smithi143.stderr: 91.0% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T07:39:57.131 INFO:teuthology.orchestra.run.smithi143.stderr: 93.1% -- replaced with /var/log/ceph/d3dfd926-700e-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T07:39:57.133 INFO:teuthology.orchestra.run.smithi143.stderr: 2024-09-11T07:39:57.134 INFO:teuthology.orchestra.run.smithi143.stderr:real 0m0.568s 2024-09-11T07:39:57.134 INFO:teuthology.orchestra.run.smithi143.stderr:user 0m2.799s 2024-09-11T07:39:57.134 INFO:teuthology.orchestra.run.smithi143.stderr:sys 0m0.127s 2024-09-11T07:39:57.134 INFO:tasks.cephadm:Archiving logs... 2024-09-11T07:39:57.134 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901214/remote/smithi143/log 2024-09-11T07:39:57.135 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T07:39:57.544 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901214/remote/smithi164/log 2024-09-11T07:39:57.545 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T07:39:57.808 INFO:tasks.cephadm:Removing cluster... 2024-09-11T07:39:57.808 DEBUG:teuthology.orchestra.run.smithi143:> sudo cephadm rm-cluster --fsid d3dfd926-700e-11ef-bcea-c7b262605968 --force 2024-09-11T07:39:58.280 DEBUG:teuthology.orchestra.run.smithi164:> sudo cephadm rm-cluster --fsid d3dfd926-700e-11ef-bcea-c7b262605968 --force 2024-09-11T07:39:58.751 INFO:tasks.cephadm:Teardown complete 2024-09-11T07:39:58.751 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T07:39:58.761 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T07:39:58.761 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T07:39:58.764 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T07:39:58.844 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T07:39:58.857 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T07:39:58.857 DEBUG:teuthology.orchestra.run.smithi143:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T07:39:58.860 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T07:39:58.871 INFO:teuthology.orchestra.run.smithi143.stderr:bash: line 1: ntpq: command not found 2024-09-11T07:39:58.874 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-09-11T07:39:58.875 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T07:39:58.875 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2024-09-11T07:39:58.876 INFO:teuthology.orchestra.run.smithi143.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 46 +183us[ +192us] +/- 62ms 2024-09-11T07:39:58.876 INFO:teuthology.orchestra.run.smithi143.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 +54us[ +62us] +/- 65ms 2024-09-11T07:39:58.876 INFO:teuthology.orchestra.run.smithi143.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 -217us[ -217us] +/- 63ms 2024-09-11T07:39:58.876 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:39:58.879 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T07:39:58.879 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-09-11T07:39:58.879 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 42 +179us[ +179us] +/- 62ms 2024-09-11T07:39:58.879 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv02.front.sepia.ceph.com 3 6 377 43 +70us[ +93us] +/- 65ms 2024-09-11T07:39:58.879 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 -206us[ -206us] +/- 63ms 2024-09-11T07:39:58.879 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:39:58.879 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T07:39:58.890 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T07:39:58.891 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T07:39:58.923 DEBUG:teuthology.orchestra.run.smithi143:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T07:39:59.000 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T07:39:59.073 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:39:59.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:59.102 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:39:59.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:59.131 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T07:39:59.140 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T07:39:59.182 INFO:teuthology.task.internal:Duration was 1309.336557 seconds 2024-09-11T07:39:59.183 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T07:39:59.215 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T07:39:59.215 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T07:39:59.218 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T07:39:59.254 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:39:59.257 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:39:59.591 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T07:39:59.591 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi143.front.sepia.ceph.com 2024-09-11T07:39:59.592 DEBUG:teuthology.orchestra.run.smithi143:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T07:39:59.653 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-09-11T07:39:59.653 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T07:39:59.684 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T07:39:59.684 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T07:39:59.696 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T07:39:59.751 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T07:39:59.751 DEBUG:teuthology.orchestra.run.smithi143:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T07:39:59.763 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T07:40:00.038 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T07:40:00.047 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T07:40:00.048 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T07:40:00.073 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T07:40:00.099 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T07:40:00.110 DEBUG:teuthology.orchestra.run.smithi143:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T07:40:00.116 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T07:40:00.137 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = core 2024-09-11T07:40:00.163 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-09-11T07:40:00.181 DEBUG:teuthology.orchestra.run.smithi143:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T07:40:00.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:00.211 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T07:40:00.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:00.235 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T07:40:00.245 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T07:40:00.246 DEBUG:teuthology.misc:Transferring archived files from smithi143:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901214/remote/smithi143 2024-09-11T07:40:00.246 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T07:40:00.297 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901214/remote/smithi164 2024-09-11T07:40:00.297 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T07:40:00.333 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T07:40:00.333 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T07:40:00.336 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T07:40:00.383 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T07:40:00.394 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T07:40:00.395 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T07:40:00.426 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T07:40:00.426 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T07:40:00.428 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T07:40:00.441 INFO:teuthology.orchestra.run.smithi143.stdout: 265889 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 07:40 /home/ubuntu/cephtest 2024-09-11T07:40:00.443 INFO:teuthology.orchestra.run.smithi164.stdout: 265329 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 07:40 /home/ubuntu/cephtest 2024-09-11T07:40:00.445 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T07:40:00.459 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T07:40:00.527 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} duration: 1309.33655667305 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-11T07:40:00.527 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T07:40:00.611 INFO:teuthology.run:pass